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Systems  Development  (IPISD),  Technical  Level  Wos*kshop.  The  Technical  Level 


Workshop  is  one  part  of  a canplete  package  of  materials  which  make  up  the 
entire  Interservice  Procedures  program.  In  the  program,  there  are  three 
levels  of  workshops: 

The  Technical  Level  Workshop  which  is  intended  for  those  indivi- 


duals who  will  actually  be  performing  the  tasks  associated  with 
instructional  systems  development  (ISD)  within  the  military. 

2.  The  Managers  Workshop  which  is  intended  for  those  who  will  be 
managing  the  efforts  of  technical  level  personnel  in  the  perfor- 
mance of  ISD  duties. 

3.  The  Senior  Managers  Workshop  which  is  intended  for  senior  level 


personnel  including  School  Commandants  and  top  echelon  managers 
who  are  concerned  with  the  higher  level  management  questions  arising 
from  the  use  of  ISD  procedures. 

The  foT<Oh'1ng  b^sic  components  are  included  in  the  Technical  Level 
Wo>*ksh.>p  packoge: 

1.  Inter:>ervice  Procedures  for  Instructional  Systems  Development 
Manuals:  The  Executive  Summary  and  Model  Manua'*  provides  an  over- 
view of  tne  procedures,  and  outputs  of  all  phases  of  IPISD. 
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The  detailed  procedures  and  reference  information  for  IP ISP 
Phases  I-V  are  given  In  four  other  manuals. 

2.  Student  Vforkbook:  The  Workbook  contains  general  Instructions  to 
the  student  and  12  practical  exercise  modules.  Each  module  consists 
of  a series  of  exercises  covering  a particular  aspect  of  IPISD.  The 
Initial  exercises  have  sample  school  solutions  which  are  given  to 
the  student  following  the  completion  of  each  exerclsp.  Student 
progress  Is  evaluated  on  the  last  few  exercises  which  require  the 
student  to  prepare  a product. 

3.  IPISD  Slide/Tape  Presentation;  These  materials  which  may  be  used  in 
group  presentations  provide  Introductory  and  gene»*al  Information 
concerning  IPISD. 

4.  Workshop  Director's  Guide;  This  guide  includes  organizational  and 
administrative  Information  for  personnel  who  will  be  directing  the 
Technical  Level  Workshop.  It  also  contains  the  multiple-choice 
pre/posttests  and  answer  keys  (Appendix  A),  school  solutions  and 
feedback  for  all  exercises  (Exercises  and  Feedback),  and  forms  and 
sample  questionnaires  for  use  in  the  workshop  (Appendices  B and  C). 
The  exercises  (modules)  contained  in  the  Student  Workbook  are  not 
Included  In  the  Guide,  but  the  Workshop  Director  should  insert 
copies  In  the  Exercise  and  Feedback  Section  for  convenience  In  use. 
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Role  of  the  Workshop  Director 

Normally,  Individuals  selected  to  become  Workshop  Directors  will  have 
completed  training  In  IPI3D  under  the  auspices  of  the  Senior  Training  Officer 
within  his  service.  These  s^crkshops  .Tiay  be  held  at  local  schools.  Installa- 
tions, or  they  may  be  conducted  at  remote  sites  which  are  accessible  to  a 
sufficient  number  of  future  Workshop  Directors. 

The  IPISD  Workshop  Director  has  a key  role  In  the  Implementation  of  ISD 
procedures  In  any  command  or  Institution.  The  role  of  the  Workshop  Director 
f Is  probably  best  described  as  being  the  manager  of  certain  Instructional 
resources  with  a goal  of  Increasing  knowledges  and  skills  In  IPISD.  While 
fundamental  technical  knowledge  of  the  contents  of  the  IPISD  Manuals  Is  a 
critical  first  step  In  becoming  a Workshop  Director,  equally  Important  Is 
learning  the  approaches  and  procedures  which  have  been  worked  out  through  a 
r.vnnber  of  tryouts,  evaluations  and  revisions  In  a variety  of  military  settings. 

The  Workshop  Director's  function  Is  that  of  arranging  for  workshops  to  be 
conducted,  providing  the  necessary  facilities  and  equipment,  scheduling  the 
workshop  to  meet  the  needs  of  the  local  conmands,  and  conducting  evaluation 
and  follow-up  studies  in  order  to  Improve  the  results  of  the  workshop  in  the 
local  situation.  The  materials  have  been  designed  and  organized  so  that  the 
principal  effort  is  on  the  part  of  the  student  rather  than  a platform  pre- 
sentation by  the  Workshop  Director. 

Role  of  the  Enablers 

i The  Workshop  Enablers  are  assistants  to  the  Workshop  Director  and  serve 
as  facilitators  of  student  learning.  The  majority  of  their  time  Is  spent 
evaluating  and  providing  feedback  on  the  progress  of  students  as  t.hey  move 
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from  point  to  point  in  the  workshop.  The  Enablers  evaluate  student  products, 
answer  student  questions,  and  are  available  at  all  tiiiies  while  the  students 
are  working  with  the  manuals  or  the  practical  exercises  so  that  a smooth 
flow  of  activity  can  be  maintained. 

The  Workshop  Director  should  be  able  to  perform  any  of  the  Enabler  activ- 
ities when  required.  If  the  number  of  students  taking  the  Instruction  is 
Quite  small  (1-5),  then  it  may  be  more  efficient  for  the  Workshop  Director 
to  serve  as  the  Enabler.  In  cases  where  there  are  larger  numbers  of  students, 
however,  the  role  of  the  Workshop  Director  becomes  more  administrative  in 
nature. 


Functions  and  Local  Organizations 

The  IPISD  Manuals  have  been  divided  into  a number  of  separate  functions 
or  "Phases"  ari  the  procedures  required  to  carry  cut  these  functions  have 

t 

been  specified.  However,  these  functions  are  required  as  a part  of  the  ISD 
effort  and  tre  not  Intended  to  be  suggested  organizational  charts  for  local 
confnands.  While  there  may  be  organizational  structures  which  would  facilitate 
conducting  the  various  functions,  this  structure  is  best  left  to  local  organi- 
zations who  are  fa'*^’liar  with  their  own  problems  and  resources.  The  view 
taken  here  is  that  the  various  functions  can  be  carried  out  in  a wide  variety 
of  organizational  settings,  and  until  more  experience  is  gained  in  the 
application  of  IPIoD,  n.j  suggestions  should  be  made  about  the  required 
organizational  structure. 

Expected  Outcomes 

Tha  Technical  Level  Workshop  is  intended  to  accOTiplish  the  objectives 
that  are  set  forth  in  the  Student  Workbook.  The  level  of  skill  that 
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Individuals  going  throu^n  the  workshop  will  gain  should  enable  than  to 
perform  at  an  entry  level  job  In  a supervised  organization  charged  with 
carrying  out  the  functions  described  In  the  manuals.  That  Is,  Individuals 
who  go  through  Blocks  I.l  and  1.2  should  be  able  to  work  In  a task  analysis 
branch  under  the  supervision  of  Individuals  who  have  appropriate  supervisory 
skills  to  be  managing  such  efforts.  It  Is  not  expected  that  the  Technical 
Level  Workshop  would  produce  "experts"  In  the  ISD  procedures.  In  older  to 
achieve  expert  status,  Individuals  who  have  completed  the  workshop  should 
work  In  a supervised,  on-the-job  setting  for  a suitable  time  following  the 
workshop  before  they  are  required  to  perform  Independently  any  of  the  tasks 
taught  In  the  workshop.  By  the  time  Individuals  become  Workshop  Directors, 
they  will  be  much  more  familiar  with  the  level  of  knowledge  and  skill  attain- 
able In  the  period  available  for  training  in  IPISD.  The  Workshop  Director 
should  communicate  this  level  of  skill  to  the  managers  of  the  various  func= 
tions  so  that  they  will  not  have  unrealistically  high  expectations  of  In 
Individuals  who  have  been  sent  to  the  workshop. 


SUGGESTED  APPROACH 


The  materials  have  been  designed  so  that  the  Student  Workbook  can  be 
adapted  to  a completely  self-paced  basis.  Depending  on  the  needs  of  a 
local  command.  Individuals  sent  to  a workshop  can  go  through  the  entire 
Workbook  and  perform  according  to  standards  established  locally.  While  It 
is  possible  to  send  all  individuals  through  all  phases  of  the  material  and 
have  them  achieve  high  standards  on  all  of  the  materials,  it  Is  more  likely 
that  local  requirements  will  be  best  served  by  sending  Individuals  through 
those  blocks  most  directly  related  to  their  personal  assignments.  It  may 


6 


-A 


be  that  a local  conmand  Mould  prefer  to  send  people  only  through  Phase  I 
of  the  workshop  and  then  present  them  with  an  overview  of  the  other  phases. 

On  the  other  hand»  individuals  who  are  going  to  work  on  the  evaluation 
function  will  have  to  have  thorough  and  first  hand  knowledge  about  all  of 
the  procedui-es  contained  in  the  manuals. 

Because  of  the  modular  nature  of  the  IPISD  materials  and  the  various 
needs  and  structures  of  different  commands,  six  options  for  modifying  the 
content  of  the  workshop  have  been  outlined  in  Table  1.  These  are  the  only 
options  that  are  recommended,  hcwever,  since  any  other  combination  would 
reduce  the  effectiveness  of  the  implementation  of  the  IPISD  Model ♦ For 
example,  if  ^n  individual  responsible  for  developing  instructional  materials 
attempted  to  take  on  the  Phase  III  materials,  it  is  not  ’ikely  that  he  would 
have  complete  and  effective  communication  with  personnel  in  the  task  analysis 
group  or  have  the  necessary  understanding  of  their  products 

The  Workshop  Director  should  carefully  lay  oCit  individual  requirements 
and  provide  adequate  time  and  support  to  insure  that  the  individuals  going 
through  the  workshop  are  properl>  trained  for  their  own  organizational 
requirements. 

In  addition  to  the  workshop  content  options,  the  Workshop  Director  may 
also  select  different  options  for  administration.  Depending  on  the  needs 
and  priorities  of  t'^tv  l‘’’ca1  command,  the  Workshop  Director  may  train  students 
in  relatively  large  groups  or  in  smaller  numbers  on  a continuous  basis-  For 
example,  the  Worksh«>  . rector  could  conduct  a formal  workshop  after  50 
st  idents  have  been  le^ic'-ated,  or  he  could  conduct  the  workshop  continuously 
( /sr  several  months  win*  only  a few  students  participating  at  any  one  time. 
For  planning  the  adi.'.nistration  of  a workshop,  the  average  full-time  student 


s 

with  proper  Enabler  support  (1:5)  can  complete  the  12  module  workshop  in 
10  working  days.  Depending  on  the  number  of  Enablers  and  the  students' 
backgrounds,  this  time  my  be  a few  days  longer  or  shorter. 

Mhere  possible,  tiie  Workshop  Director  should  be  thoroughly  familiar 
with  ttie  various  departments  or  organizations  from  which  the  students  will 
come.  In  addition,  he  should  have  a good  understanding  of  the  department's 
expectations  for  the  student  when  he  returns  to  cuty.  It  has  been  found 
that  one  of  the  more  confusing  and  unsettling  parts  of  the  Workshop  Director's 
job  is  that  of  trying  to  explain  to  the  students  how  they  will  use  what  they 
have  learned  when  they  return  to  regular  duty.  In  many  cases,  of  course, 
their  future  assignments  will  not  be  con^letely  known  ?t  the  time  they  atterxl 
the  workshop.  Wherever  possible,  though,  the  Workshop  Director  should  initi- 
ate  questions  in  order  to  be  able  to  explain  clearly  to  the  student  iWiat  he 
will  be  expected  to  ^ tdten  he  returns  to  duty. 

By  interviewing  the  managers  of  departments  and  organizations  that  will 
be  sending  students  to  tt^  workshop,  the  Vk)rkshop  Director  will  be  able  to 
make  a preliminary  determination  of  the  workshop  options  most  appropriate 
for  various  students.  This  initial  selection  of  options  sN)uld  be  confirmed 
through  interviews  with  the  students  and  evaluations  of  their  backgrounds 
and  experience.  (See  s^ple  forms.  Appendix  8.) 

The  Workshop  Director  will  normally  be  a subject  matter  expert  on  some 
aspect  of  the  training  establishment  in  which  he  works.  But  as  Workshop 
Director,  that  will  not  be  his  most  important  role.  He  will  be  the  subject 
Blatter  expert  in  the  ISO  procedures  which  are  the  content  of  the  workshop. 
However,  the  itudtnts  COTinq  to  the  workshop  should  be  subject  matter 
specialists  in  a field  where  they  will  be  expected  to  work  when  they  return. 
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, One  of  the  purposes  of  the  vrarkshop  is  to  teach  Individuals  who  are 

specialists  in  an  occupation  to  apply  IPISD  to  that  occupation.  It  is  not 
expected  that  they  would  be  able  to  teach  others  how  to  do  ISO. 

• Prior  to  a workshop*  the  students  who  will  be  attending  should  be  sent 

a meaorandum  which  explains  the  purposes  and  procedures  of  the  workshop  and 
, i describes  the  materials  that  they  should  bring  with  them.  In  addition,  the 
students  should  also  be  provided  with  the  IPISD  Manuals  to  permit  advance 
preparation. 

When  the  student  comes  to  the  workshop  he  should  bring  with  him  the 

, I 

necessary  occupational  data,  task  analysis  data,  technical  manuals  and 
reference  materials  which  will  be  used  throughout  the  v'orkshop  as  the  student 
generates  the  required  workshop  products.  While  some  occupational  specialties 
will  be  somewhat  more  directly  translatable  into  ISD  procedures  because  of 
their  job  structures,  it  is  critical  that  students  bring  with  them  materials 
on  which  to  work.  Within  a given  joo  structure,  the  materials  selected  should 
be  as  simple  as  possible.  It  is  more  difficult  and  time  consuming  to  illus- 
trate ISD  concepts  and  provide  feedback  for  extremely  technical  or  complex 
tasks  or  duties. 

It  is  intended  that  the  ultimate  configuration  of  the  IPISD  Technical 

% 

ji  Level  Workshop  will  not  be  achieved  until  local  commands  h?ve  had  experience 

I in  using  it  with  a variety  of  trainees.  After  experience  has  been  gained, 

I • much  more  attention  can  be  paid  to  adapting  the  procedures  to  meet  local 

needs  and  conditions.  It  is  suggested  that  the  outlined  procsdtres  contained 
in  this  guide  be  followed  as  closely  as  possible  until  sufficient  experience 

has  been  gained  to  make  each  local  adaptation  an  improvesnent  rather  than  an 
unnecessary  change.  As  an  aid  in  organizing  and  conducting  the  workshop, 
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an  IPISD  Workshop  Director's  Checklist  has  been  provided  in  Appendix  D. 

During  Phase  III  of  the  workshop,  students  are  asked  to  generate  some 
sample  learning  materials.  It  is  usually  advantageous  to  have  ai^ailable 
appropriate  trainees  for  the  workshop  participants  to  use  in  trying  out  ' 
their  test  materials.  In  some  instances,  these  materials  can  be  tried  out 
on  other  students  in  the  workshop,  but  this  is  not  necessarily  the  best 
use  of  their  time.  Where  possible,  the  Workshop  Director  should  arrange 
for  appropriate  members  of  the  target  population  to  be  available  for  materials 
tryout  during  that  period.  This  should  not  be  difficult  when  the  workshop  is 
conducted  at  schools  for  their  own  personnel. 

FACILITIES,  EQUIPMENT,  AND  PERSONNEL 


Facilities 

Through  time,  it  has  been  learned  that  Adequate  facilities  are  very 
important  for  the  smooth  flow  and  function  of  the  IPISD  Technical  Level 
Workshop.  And,  while  the  nature  of  the  work  does  not  require  any  dedi dated 
or  peculiarly  adapted  workspace,  the  following  reconimenoations  should  be 
quite  helpful. 

Each  parMcipant  should  have  enough  desk  space  so  that  he  can  lay  out 
his  IPISD  Manuals,  Student  Workbook,  and  working  materials  in  such  a way 
as  to  have  reasonable  access  to  them  at  all  times.  In  some  instances, 
carrels  can  be  used  when  available.  In  other  instances,  regular  work  tables 
can  be  used  provided  that  there  is  enough  space  for  each  student  to  spread 
his  materials  out  and  work  comfortably. 

A second  requirement  is  that  there  should  be  a specific  place,  reason- 
ably isolated  from  the  rest  of  the  workshop,  for  the  conduct  of  testing'. 
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There  are  two  kinds  of  tests  administered  to  the  student  during  the  workshop. 
One  of  these  is  a multiple-choice  test  which  serves  as  a management  tool  to 
guide  the  students  through  the  manuals,  and  the  other  is  c performance  test 
in  the  form  of  practical  exercises  contained  in  the  Workbook  which  the 
Enabler  will  need  to  evaluate  and  discuss  with  the  student. 

It  has  also  been  found  helpful  for  the  students  have  a separate  area 
nearby  where  they  can  discuss  the  materials  being  learned.  Typically,  small 
groups  of  people  working  on  the  same  modules  will  normally  find  that  they 
have  coimnon  problems  and  that  they  can  learn  many  things  from  other  students. 
The  best  arrangement  is  to  have  this  discussion  area  far  enough  away  from 
the  work  area  so  that  lively  discussions  will  not  disturb  those  who  are  con- 
centrating on  specific  problems  at  the  moment.  The  workshop  area  should  be 
kept  as  quite  and  orderly  as  possible  since  a large  amount  of  time  must  be 
devoted  to  reading  and  studying  the  manuals  and  the  preparation  of  materials 
as  a pafct  of  the  exercises.  The  workrooms  should  be  conducive  to  work. 

It  is  also  very  helpful  to  have  a separate  space  easily  accessible  to 
the  students  where  they  can  go  to  find  the  Workshop  Enablers.  The  Workshop 
Enablers  spend  a great  deal  of  time  evaluating  products  and  providing  indi- 
vidual feedback  to  the  student.  In  each  phase  of  the  workshop,  there  are 
frequent  checkpoints  for  Enabler  and  student  discussions. 


Equipment 

The  only  equipment  required  by  the  IPISD  program  is  some  device  capable 
of  playirg  slide/ tape  presentations.  A part  of  the  Technical  Level  Workshot: 
package  is  a pulsed  S^ide/Tape  Presentation  which  provides  a brief  overview 
of  the  IPISD  Model  and  examples  cf  high  pay-off  ISD. 


other  than  the  slide/tape  device,  the  only  requirements  would  be 
naturally  satisfied  by  supplying  normal  coursework  materials  including 
paper,  pencils,  and  other  necessary  and  associated  items. 

Personnel 

Several  weeks  in  advance  of  the  first  workshop,  the  Workshop  Director 
must  select  the  personnel  who  will  be  serving  as  the  Enablers  throughout 
the  workshop.  If  these  individuals  have  not  had  prior  training  in  IPISD, 
the  Workshop  Director  should  conduct  the  complete  Technical  Level  Workshop 
for  them  and  insure  that  they  are  thoroughly  knowledgeable  not  only  con- 
cerning the  workshop  content,  but  also  all  administrative  procedures.  In 
order  to  be  an  effective  Enabler,  the  ir.dividual  must  be  an  honor  graduate 
of  the  workshop  and  a subject  matter  expert  in  the  military  specialties  that 
win  be  using  IPISD. 

The  number  of  Enablers  required  will  depend  on  the  number  of  students 
in  the  workshop.  Previous  experience  in  acini nistering  the  workshop  indicated 
that  a ratio  of  Enablers  to  students  of  1:5  would  be  a minimum  number  of 
Enablers.  Considering  the  time  requir’d  to  evaluate  products  and  provide 
feedback,  a ratio  less  than  1:5  would  senuu«^ly  >"°'<'ice  the  effectiveness  of 
the  Enablers  or  cause  the  students  to  waste  working  time  in  queues. 

USES  OF  WORKSHOP  MATERIALS 

The  workshop  materials  are  built  around  the  IPISD  Manuals.  The  manuals 
are  divided  into  five  separate  Phases  and  there  is  an  Executive  Summary 
volume  which  presents  an  overview  of  the  entire  set  of  procedures.  The 
manuals  are  intended  primarily  to  be  for  reference  and  guidance  purpo'.es. 
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Individuals  who  will  perform  the  various  tasks  associated  with  IPISD  on 
the  job  must  have  access  to  the  manuals  on  a continuing  basfs.  It  is  the 
responsibility  of  the  Workshop  Director  to  become  thoroughly  famfliar  with 
all  of  the  content  and  procedures  in  the  manuals  prior  to  the  workshop. 

In  addition  to  the  reference  manuals,  there  is  a Student  Workbook  which 
is  the  principal  organizing  document  for  instruction  in  the  use  of  the  manuals, 
the  Workbook  contains  a number  of  questions  to  be  answered,  exercises  to  be 
worked  with  school  solutions,  and  exercises  to  be  worked  which  the  Enabler 
must  judge.  In  combination  with  the  student  supplied  data  manuals  and  in- 
formation that  he  brings  from  his  job,  the  Workbook  provides  training  in  the 
knowledges  and  skills  necessary  to  perform  IPISD  tasks. 

Prior  to  the  workshop,  the  Workshop  Director  should  determine  the  most 
appropriate  workshop  content  option  for  each  student.  Based  on  a student's 
assignment  and  the  anticipated  requirement  tor  skills  that  may  be  acquired 
in  the  workshop,  the  Workshop  Director  should  identify  those  IPISD  phases 
'^iiich  the  student  should  master  and  those  phases  which  he  should  become 
fo.p.lliar  with  in  order  to  understand  the  entire  model.  (See  sample  question- 
naire, Job  Profile  Form,  Appendix  5.)  For  the  IPISD  phases  considered 
essential  for  subsequent  job  performance  and  selected  for  mastery,  the  student 
should  obtain  a score  of  iOOS  on  all  the  tests  for  those  phases.  Past 
experience  has  indicated  that  for  those  people  who  are  going  to  work  on  the 
activities  of  a given  phase,  a score  of  100%  on  the  posttest  is  a reasonable 
expectation.  For  the  IPISD  phases  designated  for  familiarity,  the  student 
is  required  only  to  read  the  tests,  practical  exercises,  and  the  Executive 
SuiTwary  sections  on  those  phases. 
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The  imjl tipi  e-choice  tests  which  are  used  as  the  pre/posttests  are 


provided  In  Appendix  A.  These  tests  have  a very  specific  purpose:  To 


Insure  that  each  trainee  has  read  and  studied  the  content  of  the  manuals 


prior  to  the  time  that  he  attempts  to  do  the  exercises.  In  the  front  of 


the  Workbook  there  Is  a description  and  flowchart  which  Indicates  to  the 


student  how  he  Is  to  proceed  through  the  workshop.  This  flowchart  Is  re- 


produced on  the  following  page. 


The  recommended  procedure  is  that  the  student,  upon  beginning  the  work- 


shop, take  all  of  the  Phase  I pretest.  Modules  1 through  4i  (providing  that 


the  student  Is  going  to  do  all  of  the  work  In  Phase  I),  and  have  those 


pretests  scored  by  the  Enabler.  This  is  block  1 of  the  flowchart.  Take 


Phase  Pretest,  and  Is, where  the  student  will  begin  work  on  each  phase  he 


is  to  master.  The  prestests  cover  an  entire  phase  of  the  IP ISP  Manuals  and 


are  divided  Into  units  covering  each  workshop  module  for  that  particular 


phase, 


After  the  Enabler  has  scored  the  pretest.  In  this  case  for  Phase  I,  he 


informs  the  student  of  his  score  on  the  section  that  covers  the  first  module 


of  that  phase  (Module  1).  As  described  In  the  instructions  in  the  Workbook, 


s student  who  reaches  his  criterion  on  the  pretest  is  assumed  to  have  mastered 


the  contents  of  that  section  of  the  manuals.  He  is  not  required  to  do  the 


I'eadings  and  is  given  the  module  exercises  immediately  after  receiving  his 
pretest  score  (block  2 of  the  flowchart). 


If  the  student  has  not  reached  his  criterion  for  that  module  on  the 


pretest,  the  Enabler  gives  him  the  reading  assignmMt  in  the  manuals. (bf#ck  3 
of  the  flowchart).  For  Module  1 the  reading  assignsjent  Is  Blocks  1.1  and  1,2. 
of  the  manuals.  After  the  student  has  co.mpletad  the  reading  and  has  studio 
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the  terms  and  concepts,  he  then  requests  the  postt  ' ■ >r  Module  1 (block 
4 of  the  flowchart).  The  posttest  is  that  ;»r<»test  covering 

the  module  that  the  student  will  be  working  on.  m^ets  his  criterion 

he  is  then  given  the  Module  1 exercises.  It  is  imp.  .'tant  that  the  student 
not  be  given  the  exercises  prior  to  the  time  that  ’ re?d  completely 
through  his  re.'ding  assignment.  Past  exoerience  ha.  indicated  that  some 
students  will  take  the  exercises  and  try  to  look  up  the  answers  in  the 
text  without  understanding  the  total  context. 

When  students  do  not  meet  the  criteria  on  the  posttest,  the  Workshop 
Director  or  Enabler  must  decide  whether  to  discuss  the  erroneous  answers 
with  him  or  have  him  read  the  assignment  again  and  retake  the  test  prior 
to  giving  him  the  exercises.  The  former  approach  worked  very  satisfactorily 
for  most  students. 

Since  the  tests  are  used  for  management  purposes,  it  is  extremely  im- 
portant to  maintain  security  of  all  tests  in  the  workshop. 

The  exercises  each  module  are  usually  sequenced  as  follows: 


Problem  with 
required 
information 
supplied  


Enabler  Problem  with  ’ Enabler's  Questions 

gives  ^ student  L ^evaluation on  other 

school  ^ supplied  ^ topics  in 

solution!  information  module 


After  the  student  has  cor-spleted  the  initial  exercise  in  Module  1,  he 
brings  hi  - -^rk  to  the  inabler  who  checks  it  off  as  acceptable  or  unacceptable 
(block  5 of  the  flowcha.  i,) . If  it  is  unacceptable,  the  student  is  given 
guldsnce  and  asked  to  correct  it-.  For  all  guidance  and  feedback  given  by 
the  Enabi&r,  the  emphasis  should  be  on  providing  positive  reinforcement  as 
a technique  for  directing  student  activities  and  developing  areas  of  perfor- 
mance when  proficiency  is  low. 


“’"■r  ' “ .1-  I'' 


Suggested  guidance  and  feedback  for  the  Enabler  to  give  the  student  Is 
provided  In  the  Exercises  and  Feedback  section  of  this  volume  (page  23). 

School  solutions  or  sample  answers  are  provided  for  each  of  the  questions 
and  exercises  in  each  of  the  modules.  The  Workshop  Director,  of  course,  will 
have  gone  through  a workshop  himself  prior  to  the  time  that  he  is  ronducting 
one  and  will  have  a much  better  understanding  of  the  requirements  and  standards 
that  the  student  Is  expected  to  meet. 

After  the  student  has  performed  all  of  the  exercises  In  Module  1,  he  Is 
then  ready  to  proceed  to  the  next  step.  He  Is  given  his  pretest  score  for 
the  next  module  aH  Is  given  the  exercises  if  he  has  reached  criterion.  If 
he  has  not  reached  criterion,  he  begins  rending  the  material  for  the  next 
module  and  ther  takes  the  posttest  for  that  module.  As  soon  as  he  meets  the 
standards  on  the  posttest  he  will  br  ready  to  begin  the  exercises.  This 
procedure  is  repeated  until  the  student  has  completed  his  final  module  in 
the  workshop. 

Appendix  C contains  copies  of  sample  forms  that  have  been  found  useful 
to  the  students  In  the  conduct  of  the  workshop.  These  forms  can  be  used  as 
masters;  the  Workshop  Director  should  insure  that  there  are  ample  copies 
available  so  that  the  students  will  have  them  when  they  are  called  for  in 
the  workshop  exercises.  Whenever  possible,  exact  copies  of  these  forms 
should  be  used  In  the  workshop  to  insure  uniformity  of  products  and  adherence 
to  ISD  principles  and  procedures  and  to  facilitate  the  enabling  process. 

These  forms,  however,  were  designed  specifically  for  use  in  the  workshop  and 
they  may  be  used  as  considered  appropriate  during  normal  activities  and 


operations. 
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The  Student  should  retain  the  exercises  that  he  worl'.s  and  the  forms 
that  he  fills  out  during  the  vmrkshop  for  use  as  reference  material  when 
he  returns  to  his  job.  On  the  job.  he  should  have  access  to  a complete 
set  of  IPISD  Manuals  or  those  manuals  related  to  that  part  of  the  process 
for  which  he  Is  responsible.  He  should  retain  possession  of  his  Workbook 
so  that  any  questions  that  come  up  about  procedure  will  be  more  readily 
answered  through  the  availability  cf  convenient  references. 

One  of  the  more  difficult  jobs  the  Enabler  will  have  Is  scoring  perfor- 
mance tests  In  those  areas  where  he  Is  not  a subject  matter  exp-irt.  As 
indicated  earlier,  the  performance  tests  are  the  final  exercises  of  a module. 
Many  times,  the  Enabler,  being  unfamiliar  with  the  field  in  Wfilch  the  student 
is  working,  will  have  to  Interview  the  student  to  make  certain  that  the  lay- 
out and  organization  of  the  work  Is  conceptually  consistent  with  the  procedures 
outlined  In  the  manuals.  Judgment  will  be  required  In  determining  exactly  how 
much  detail  and  precision  a student  must  present  In  order  to  receive  a pass 
on  the  performance  test.  Previous  experience  in  IPISD  workshops  will  be  of 
great  value  to  the  Enabler  in  deciding  vdiether  or  not  student  performance 
meets  acceotable  standards. 

LOCAL  ADAPTATIONS 

During  the  development  of  the  IPISD  project  It  was  found  that  one  of  the 
problems  of  greatest  concern  to  the  student  is  what  he  Is  going  to  be  expected 
to  do  when  he  returns  to  his  job.  Often,  the  Workshop  Director  will  be  able 
to  provide  this  Information  since  he  will  have  discussed  the  workshop  with  the 
trainee's  supervisor.  Supervisors  will  be  very  much  concerned  about  the 


I ! 
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amount  of  time  required  for  the  student  to  complete  the  training.  The 
amount  of  time  devoted  to  the  workshop  can  be  adjusted  somewhat  by  the 
Workshop  Director  depending  on  the  needs  of  the  comnand  and  the  expected 
performance  of  the  student  when  he  returns  to  work.  As  indicated  earlier, 
it  is  expected  that  the  student  would  return  to  a Job  where  he  can  receive 
supervision  and  assistance  in  the  performance  of  ISD  procedures. 

Through  time,  the  Workshop  Director  should  collect  an  impressive  array 
of  student  products  which  can  then  be  used  as  examples  of  the  kind  of  work 
expected  at  the  local  command.  Since  these  products  will  all  deal  with 
DOSs  of  concern  to  the  local  command,  they  will  be  relevant  and  more  readily 
understood  by  the  student  than  the  examples  provided  in  the  IPISD  packages. 
It  will  probably  be  useful  to  the  student  if  the  Workshop  Director  and  the 
Enablers  go  through  those  exercises,  have  '•.hem  typed  up  and  organized,  and 
then  highlight  o<*  emphasize  significant  features  which  make  them  superior 
products.  These  kinds  of  examples  can  have  a beneficial  effect  on  the 
student  as  he  progresses  tr rough  the  workshop. 

It  is  also  useful  if  the  Workshop  Director  can  have  follow-up  data 
collected  on  performance  of  students  once  they  have  returned  to  their  jobs. 
In  this  way,  the  Workshop  Director  can  deterr.ine,  over  a relatively  short 
period  of  time,  the  additional  emphasis  or  de-emphasis  needed  on  any  given 
topic  within  the  current  organization  of  the  workshop.  When  it  becomes 
apparent  that  materials  or  exercises  should  be  added,  the  Workshop  Director 
can  do  this  and  Improve  the  effectiveness  of  the  workshop  for  the  local 
c(xnmand. 

Another  important  job  that  the  Workshop  Director  and  Enablers  have  is 
collecting  reference  materials  and  providing  them  in  the  workshjp  area.  At 


the  end  of  each  phase  in  the  manuals  there  are  suggested  references  which 
go  Into  n»re  detail  and  provide  additional  approaches  to  the  solution  of 
various  {.roblems  encountered  in  the  workshop.  As  these  references  are 
accumulated,  they  can  be  used  and  woven  into  tie  workshop  so  that  those 
areas  requiring  additional  emphasis  or  more  clarification,  or  those  in 
which  the  student  has  a more  incense  interest  or  need,  can  be  explored  and 
better  products  and  instruction  will  result. 

Each  service  or  branch  of  a service  has  existing  materials  which  can 
be  useful  when  added  to  the  regular  workshop  materials.  These  would  include 
such  things  as  films,  slide/tape  presentations,  video' tapes,  audio  tapes,  and 
other  mediated  materials  which  highlight,  emphasize,  provide  examples,  or 
simply  provide  an  overview  of  given  approaches  and  procedures.  These  existing 
mediated  materials  can  add  a dimension  to  the  workshop  and  can  probably  have 
a:  i'lK'rtant  positive  effect  on  how  students  feel  about  doing  the  work. 

‘^cr  the  workshop  has  been  conducted  a number  of  times  in  a local  situa- 
tiof;,  ,10  Workshop  Director  may  wish  to  change  the  schedule.  There  are  many 
options  available  and  each  of  these  must  be  considered  when  trying  to  n«et 
.he  needs  of  local  conmands.  In  some  Instances,  it  may  be  desirable  to  use 
tiilf-day  sessions  while  the  individual  does  his  regular  job  in  the  other 
half-day.  There  are  both  advantages  and  disadvantages  to  th’s  technique 
but,  given  proper  management,  options  can  certainly  serve  to  improve  the  use- 
fulness of  the  workshop  to  the  local  command.  It  may  be  that  the  defnands  on 
the  time  of  the  individual  and  his  performance  are  so  high  that  the  Workshop 
director  will  wish  to  extend  the  length  of  time  these  particular  individuals 
spend  in  the  workshop  in  order  to  provide  training  to  a higher  standard. 
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This  Mould  occur  In  those  Instances  where  trainees  are  expected  to  perform 
more  or  less  Independently  when  they  return  to  their  jobs  or  when  they  will 
not  have  supervisors  available  who  are  knowledgeable  In  ISD.  The  Workshop 
Director  should  carefully  examine  each  of  the  options  as  the  local  require- 
ments change.  ' 

FOLLOW-UP  PROCEDURES 

One  of  the  more  important  purposes  that  the  Workshop  Director  can  serve 
Is  that  of  providing  a link  between  the  training  experience  that  Individuals 
In  the  workshop  have  and  the  kinds  of  problems  that  they  must  deal  with  when 
they  return  to  the  job.  By  talking  to  trainees  after  they  have  returned  to 
their  jobs,  and  their  supervisors,  the  Workshop  Director  will  be  able  to 
Identify  local  problems  and  problems  within  the  workshop  or  materials  and 
make  suitable  adaptations.  Student  feedback  Is  a valuable  source  of  Infor- 
mation in  deciding  on  what  kinds  of  changes  and  emphasis  are  required  in 
order  to  meet  the  needs  of  local  conmands. 

An  Important  obligation  of  the  Workshop  Director  is  that  of  transmitting 
critiques  and  suggestions  for  changes  In  the  manuals,  exercises,  or  materials, 
to  the  command  responsible  for  ISD  In  yoTtr  service.  These  critiques  and 
suggestions  are  probably  the  greatest  single  source  of  evaluation  and  revision 
data  that  can  be  provided.  No  matter  how  many  times  the  materials  have  been 
tried  out  In  various  environments,  they  will  still  not  have  been  subjected  to 
the  day-to-day,  i^k-to-week,  month-to-month  use  possible  In  local  situations. 
Therefore,  experience  with  local  use  Is  of  the  utmost  Importance  In  getting 
Improved  materials  transmitted  to  the  Interservica  community.  It  Is  hoped 
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that  Workshop  Directors  will  be  personally  responsible  for  providing  these 
reviews  and  critiques  on  a periodic  basis  so  that  future  editions  of  these 
mnuals  can  be  Improved. 

Finally,  the  Workshop  Director  should  give  careful  consideration  to  the 
issue  of  the  materials.  How  many  sets  of  IPISD  Hanuals  are  required  to 
serve  the  needs  of  the  command,  how  many  Workbooks  are  needed  for  mich  set 
of  manuals,  how  many  Workshop  Director's  Guides,  and  so  on,  are  required  in 
order  to  be  sure  that  there  are  adequate  materials  to  do  the  Job  in  question. 
While  Initial  estimates  have  been  made,  and  Issues  provided  on  those  estimates, 
only  practical  field  experience  can  provide  the  kind  of  Information  necessary 
tc  Insure  the  most  effective  and  economical  package  for  future  needs. 

Please  feel  free  to  make  any  comments  concerning  the  procedures,  materials, 
or  anything  you  see  which  might  have  an  Impact  on  the  materials.  Suggestions 
fo-'  revisions  or  changes  should  be  sent  to: 

ISD  Project 

Center  for  Educational  Technology 
lA  Tully  Building 
Florida  State  University 
Tallahassee,  FL  32306 


p 
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The  exercises  from  the  Stude  : Workbook  have  been  omitted  hare.  It  is 
suggested  that  the  Workshop  Director  insert  copies  of  the  exercises  at  the 
appropriate  places  in  this  section  for  his  convenience. 

You  will  note  a coded  page  number  in  the  upper  left-hand  corner  of  the 
Enabler  Feedback,  Modu'^es,  and  Pre/Fosttests.  These  are  designed  as  an  aid 
in  ov'ganizing  the  materials  for  printing,  collating,  and  grouping  the 
materials  for  use  by  students  and  Enablers.  An  explanation  of  this  code 
is  as  follows: 


MODULE  - TW  - Ml 


1 


ENABLER 

FEEDBACK 


t 

Technical 

Workshop 

Material 


t 

Module  # 


t 

Page  # 


Enable'^ 
Feedback 
for  Module  # 


TW  - EFl  - 1 


For  example,  TW-M3-4, 
Feedback  for  Module  3 


is  Technical  Workshop  Module  3,  page  4.  The  Enabler 
is  coded  TW-EF3  with  the  apnropriate  page  number. 
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ENABLER  FEEDBACK 
MODULE  1 


1.  All  items  must  be  correct  as  follows: 

Correctly  written  tasks:  items  2,  3,  5,  6,  9,  and  10; 

Incorrectly  written  tasks:  Items  1,  4,  7,  8 and  11. 

If  the  student  misses  any  of  t'nf.  itjmis,  ask  him  to  explain  his 
reason  for  classifying  each  wrr-ig  response,  (pp.  12-17) 


The  student  should  select  tasks  1 through  6 and  tasks  8 and  9.  He  might 
choose  10  and  11 > however,  only  31%  and  23%  of  first  yaar  patrohnen 
perform  10  and  11  respectively.  Those  tasks,  t’’arefori\  r.:'i  ptubably 
lower  priority  when  only  7 tasks  can  be  select 

If  the  student's  responses  indicate  a lack  of  understanding  of  the 
principles  involved,  «"efer  him  to  pages  136-148. 


3.  a.  The  job  definition  must  match  the  definition  given  on  pages 
8-10  of  the  Phase  I manual. 

(1)  If  the  job  is  inadequately  defined,  tell  the  student 
what  is  wrong  and  tell  him  how  to  fix  it. 

(2)  Repeat  (1)  above  until  the  item  is  correct. 

b.  The  duty  statements  must  be  written  according  to  the  guidelines 
given  on  pages  9-12  of  the  Phase  I manual.  The  Enabler  must  be 
satisfied,  based  on  his  knowledge  of  ths  job,  that  substantially 
all  the  duties  have  been  listed.  The  Enabler  may  question  the 
student  about  what  the  enabler  suspects  are  inappropriate  or 
missing  duties.  Indicate  what  is  wrong  with  the  duty  statements 
and  have  student  make  corrections.  Repeat  until  list  is 
acceptable. 

Sample  Answer 

Flight  Operations  Coordinator: 

Major  Duties— 

1.  Processing  flight  plans 

2.  Interpreting  weather  information 

3.  Disseminating  weather  information 

4.  Selecting  aircraft  for  missions 

5.  Maintaining  Notice- to-Ainnen  (NOTAM)  files 

6.  Disseminating  NOTAMs 

7.  Operating  communications  equipment 


TW-EFl-2 


8.  Maintaining  flight  records  files 

9.  Managing  airfield  operations  activities 
10  > Haintai"1ng  functional  fiU^s 

11.  Preparing  correspondence 

12.  Handling  classified,  (naterials 

(F,d.  Note-.  While  some  of  rh«  duties  appear  to  be  more  like  tasks  e.g.,  4,  5. 
6,  12,  one  would  have  to  be  familiar  with  the  job  to  be  sure,  it  is  important 
tNt  2 or  more  duties  hav2  been  identified  so  that  the  enabler  can  verify  that 
student  can  write  lists  of  jobs,  v.kies,  and  tasks.) 


4.  All  items  in  t.ie  plan  must  be  complete  Md  consistent  with  the 
assumption!  on  the  particular  job  and  command,  as  stated  by  the 
student.  A.v  rpasonable  assumptions  are  acceptable  if  they  are 
clearly  stated.  Indicate  inappropriate  items  or  incomplete  items, 
and  have  the  student  make  corrections  until  they  conform  to  the 
general  format  of  the  plan(s)  which  are  discussed  on  pages  34-83 
and  U8-136  of  your  !SD  Manual  (Phase  I manual).  Have  the  student 
study  these  sections  if  his  initifl  product  does  not  conform  to  the 
general  format  rutlined  In  the  manuals.  Be  sure  all  required  data 
are  accounted  for.  This  includes  an  original  tasFTlst,  task  list 
verificatio'i,  evaluation  criteria  for  selecting  tasks  for  training, 

^nd  aocur.-.«,  taticn  of  conditions,  cues,  and  standards.  There  is  no 
r.‘»jd  for  more  than  8 tasks. 

Sample  Answer 

Data  Collection  Plan  for  Flight  Operations  Coordinator 
ihe  plan  is  based  on  two  assumptions: 

1.  Current  task  list  is  not  available 

2.  There  are  no  constraint?  on  travel  funds 

a . Oai.a  neede*"*-- 

Task  list:  Preliminary  task  list  will  be  developed  through  a 
conmittee  of  analysts  from  my  office  and  subject  matter  experts 
(SME)  from  the  resident  school  department  providing  the  MOS 
training.  This  task  list  will  include  only  a listing  of  duties  and 
tasks. 

Form  of  the  questionnaire  will  be  as  in  Appendix  D (Phase  I Manual, 
p.  W2).  questionnaire  will  be  used  to  validate  task  lists  and 
assist  in  selecting  tasks  for  training.  Forms  for  obtaining 
biographical  data  and  administrative  instructions  will  accomDanv 
each  q.jsstionnaire. 

Job  incumbents  will  be  provided  two  copies  of  the  questionnaire- 
one  to  be  completed  by  the  incumbent  and  one  by  his  supervisor 
for  verifying  and  validating  tasks. 


a 


.a 


2 Assuming  an  acceptable  sf  return  of  questlcnnhires, 
data  win  be  analyzed  and  task  lists  revised  as  necessary  to 
finalize  a validated  task  inve»?tory. 

3.  De'Uils  of  tasks;  Conditions 5 cues,  standards  and  task  elements 
onl y~^r  tnose  tasks  selected  for  training. 

b.  Order  for  collecting  data— 

1.  Initial  task  list  containing  duties  ?nd  tasks 

2.  Use  questionnaire  for  incumbent  and  supervisor  to  verify 
and  validate  task  lists  and  obtain  some  data  to  use  as  a 
basis  for  selecting  tasks  for  training 

3.  Details  of  tasks  selected  for  training^-cues,  conditions, 
standards,  end  elements 

4.  Simroary  of  questionnaire  results 

c.  Evaluation  criteria  for  selecting  tasks  for  training— 

1.  Frequency  of  performance 

2.  Time  between  job  entry  and  task  performance 

3.  Probable  con.tequences  of  Inadequate  performance 

4.  Percent  performing 

d.  Sources  of  data— 

Appropriate  TMs,  FMs  and  SMEs  for  detail  of  tasks.  Job  incumbent 
and  supervisor.  Subject  matter  experts. 

Since  there  are  only  550  incumbents  in  this  job,  each  incumbent 
and  his  supervisor  would  be  surveyed.  Questionnaires  would  be 
administered  by  mail  since  incumbents  are  widely  dispersed  in 
small  groips. 

e.  Data  collection  forms— 

Questionnaire  - Appendix  D (Phase  I Manual,  p.  102) 

Job  Data  Worksheet  - Block  1,3,  page  87,  modified  to  include  blocks 
for  recording  Information  from  survey  for  ease  of  reference  in 
selecting  tasks  and  selecting  instructional  setting  (e.g.,  selection 
criteria  ratings,  frequency,  % performing,  consequences,  etc.) 


The  task  statements  must  be  written  accoiding  to  the  guidelines  given 
on  pages  12-17  of  the  Phase  I manual,  and  be  sufficiently  clear  that  the 
Enabler  understands  the  task  well  enough  to  judge  the  adequacy  of  the 
documentation.  If  the  initial  product  does  not  follow  the  guidelines 
given  in  the  manuals,  show  the  student  pages  12-17  of  the  Phase  I manual 
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and  help  him  rewrite  one  of  the  Inadequate  task  statements.  Let  him 
correct  the  rest  and  then  check  again.  If  the  product  still  is  unacceptable, 
point  out  the  deficiency  and  have  the  student  make  corrections. 

Sample  Answer 

Processing  flight  plans— 

1.  Inspect  DD  Form  175  for  completion  of  required  entries. 

2.  Verify  accuracy  of  fuel  on  board  entries. 

3.  Determi?'tf  f.r  DO  Fotmi  175-1.  • 

4.  Tvjnscribe  fV:  sK  plan  information  onto  flight  data  strips. 

5.  Dissemina^  f‘,  .ght  plan  information. 

Interpreting  weather  information— 

6.  Decode  riourly  sequence  report. 

7.  Record  pilot  weather  reports  (PIREPs) 

S.  Disseminate  weather  information. 


’xk'k 


6.  The  summarized  data  should  be  complete  and  in  accordance  with  the 
student's  data  collection  plan  from  Exer-'ise  4,  The  Enabler  may 
change  some  of  the  data.  Some  guidelines  «re: 

(1)  Changing  data  in  Exercise  6 so  that  the  student  must  select 
the  desired  tasks. 

(2)  Reduce  the  nwnber  of  tasks  that  can  be  trained. 

(3)  Increase  the  number  of  tasks  that  can  be  trained. 

(4)  Make  an  arbitrary  selection.  If  this  is  done,  the  student 
should  be  told  why. 


(See  Sample  Answer,  page  5.) 
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The  student  should  tie  given  a "go"  if  and  when  his  product  follows 
the  guidelines  given  in  the  manuals.  However,  the  Enabler  should 
look  ahead  to  future  exercises,  particularly  Exercise  8 of  this ’block, 
and  the  exercises  in  Blocks  II. 1 and  III. 4.  The  Enabler  can  guide  the 
selection  of  tasks  of  the  appropriate  degree  of  complexity  by  following 
the  guidelines  in  Exercise  6.  Keep  the  tasks  limited. 

Sample  Answer 

Tasks  1,  4,  6,  and  8 from  exercise  6 were  selected  for  training. 
Rationale- 

Based  on  evaluation  of  data: 

High  percentages  who  perform  these  tasks 
Consequences  of  inadequate  performance 

The  tasks  selected  are  performed  soon  after  entry  into  duty  position 

Based  on  subject  matter  experts  opinion  these  tasks  are  fairly  difficult 
to  learn,  indicating  that  FOOT  may  be  difficult. 

Proficiency  in  these  tasks  is  essential  to  the  safe  operation  and 
orderly  flow  of  air  traffic. 


The  conditions,  cues,  standards,  and  elements  of  a task  are  adequately 
documented  if  an  individual  with  no  prior  knowledge  of  the  task  can  gain, 
from  reading  the  task  documentation,  a visual  image  of  how  the  task  should 
be  performed.  If  some  tasks  prove  to  be  too  complex  to  be  documented  in 
the  intended  time,  the  Enabler  may  reconsider  sci'-e  of  the  decisions  he 
made  in  Exercise  7 above,  and  further  modify  or  reduce  tije  tasks 
selected  for  training.  Conditions,  cues,  standards,  and  eleirents  are 
discussed  on  pages  17-33  and  76-77  of  the  Phase  I manual. 

iSee  the  sample  answer  o:i  the  following  pages.) 
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9. 


Tasks  are  selected  for  trair’^ng  based  on  need  after  evaluation  using 
selected  criteria.  Since  in  most  cases  all  tasks  cannot  be  trained, 
based  on  resource  constraints  and  other  considerations,  tasks  are 
selected  that  are  most  in  need  of  training.  Management  inputs  will 
assist  you  in  determining  how  to  optimize  the  training  dollars  and 
achieve  maximum  effect  out  of  training. 


(The  answer  should  Include  the  idea  of  optimizing  training  resources 
and  training  those  tasks  that  are  most  needed.) 
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ENABLER  FEEDBACK 
MODULE  2 


1.  The  trainee-prepareci  JPMs  may  not  be  identical  to  the  school  solutions 
which  are  given  on  pages  2-4.  These  are  possible  outcomes,  which  are 
discussed  below. 

a.  The  illustrator  JPM  should  vary  little  from  the  job  except  for 
the  problem  presented. 

b.  Two  good  alternatives  are  presented  for  the  survival  task.  JPM  la 
is  the  more  performance-oriented  JPM  and  is  the  better,  but  time 
cciistraints  may  dictate  the  increased  simulation  such  as  in  Jl^  lb. 
The  information  JPM,  le.  Is  not  acceptable. 

c.  The  wheel  vehicle  mechanic's  JPM  should  have  different  initiating 
cues  from  .‘‘.he  task.  The  directions  to  the  acfaiinistrator  wil;  vary, 
but  must  be  consistent  with  the  JPM  (see  attached  example,  pages 

5 and  6). 

2.  The  student's  answer  should  include  the  following: 

Instruction  is  based  on  JPMs  for  the  same  reason  that  testing  is  con- 
ducted by  administering  JPMs.  If  testing  the  actual  task  is  too 
dangerous  or  expensive  to  be  considered,  it  is  likely  that  training 
that  same  task  is  also  too  dangerous  or  expensive  to  bo  considered. 

A second  consideration  concerns  the  evaluation  of  student  performance. 

If  instruction  is  based  on  the  JWi,  the  effectiveness  of  that  instruction 
can  be  evaluated  by  administering  the  JPM.  However,  if  instruction  is 
based  on  the  task,  but  the  task  cannot  be  tested  because  of  constraints, 
there  Is  no  way  to  evaluate  thei instruction  or  the  students  and  you  will 
have  no  way  of  knowing  whether  you  succeeded  in  training  the  task.  You 
have  to  train  what  you  test;  that  is,  the  JPM. 

For  students  who  are  unable  to  give  this  rationale,  explain  this  answer 
and  discuss  it  with  them. 


Sample  Answer 

Because  of  constraints  of  time,  money,  personnel,  facilities  and  other 
resources,  JPMs  are  used  to  develop  and  control  training  and  measure 
the  success  of  your  training  program.  Since  many  tasks  C4nnot  be 
measured  directly,  the  JW1  strikes  a compromise  between  resource  con- 
straints and  actual  Job  performance. 
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DIRECTIONS  TO  TEST  ADMINISTRATOR  FOR  THE  WHEEL  VEHICLE  MECHANICS  TASK: 


a.  Test  Objecti ve:  The  purpose  of  this  JPM  is  to  measure  the  examinee's 

ability  to  adjust  a malfunctioning  wheel  bearing  on  a ^ ton  truck. 

b.  Equipment  Required:  h ton  truck;  one-ton  hydraulic  jeck;  general 

mechanics  t:ool  box;  V square  drive  torque  wrench;  TM  9-2320-218-20. 

c.  Environment:  Indoors  or  outdoors  in  daylight  and  dry  weather. 

d.  Layout  of  Test  Area; 

1)  Vehicle  will  be  parked  Mevel  ground)  wi^'i  both  rear  wheels 
blocked  front  and  rear  and  with  the  left  front  wheelbearing  out  of 
adjustment  (loose). 

2)  Prepare  the  wheel  bearing  as  follows:  Remove  lifting  eye  and 
cotter  pin  and  loosen  flange  nut  approximately  one-half  turn. 

Then  replace  the  cotter  pin  and  lifting  eye. 

e . Test  Administration  Procedures : 

1)  Before  any  testing  is  done,  the  test  administrator  must  insure 
that  all  pre-test  preparation  has  been  done,  and  that  all  equip- 
ment and  special  tools  are  readily  available  and  in  good  working 
condition. 

2)  The  test  consists  of  a number  of  performance  measures.  To  score, 
the  test  administrator  will  check  (V')  each  performance  measure 

as  it  is  done.  A zero  (0)  will  be  entered  for  performance  measures 
that  are  not  done.  The  examinee  is  not  to  be  informed  about  his 
scores  or  how  well  he  is  doing  during  the  test.  No  prompting  is 
permitted.  The  maximum  time  for  this  test  is  25  minutes. 

3)  When  the  test  has  been  completed,  the  test  administrator  will 
return  the  equipment  to  its  originally  prepared  condition  for 
the  testing  of  other  examinees. 

4)  The  test  administrator  will  read  the  instructions  to  the  examinee. 

TEST  SITUATION  (INSTRUCTIONS  TO  EXAMINEE): 

During  a road  test  the  left  front  wheel  of  this  vehicle  wobbled  and  shook. 

We  already  know  it's  caused  by  a loose  wheel btaring.  Your  task  is  to 

repair  it.  Here  are  tools  and  reference  materials  if  you  wish  to  use  them. 

You  have  25  minutes  to  finish  this  job. 
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PERFORMANCE  MEASURES  (BEHAVIORS)  AND  SCORE  SHEET: 

1.  ( ) Obtains  jack  and  positions  it  properly. 

2.  ( ) Raises  vehicle. 

3.  ( ) Loosens  locknut  on  lifting  eye. 

4.  ( ) Removes  lifting  eye  from  wheel. 

5.  ( ) Removes  cotter  pin. 

6.  ( ) Selects  torque  wrench. 

7.  ( ) Tightens  flange  nut  to  30  Ib-ft  torque. 

8.  { ) Releases  torque  by  loosening  the  flange  nut. 

9.  ( ) Tightens  flange  nut  finger  tight. 

10.  ( ) Replaces  cotter  pin. 

11.  ( ) Replaces  lifting  eye. 

12.  ( ) Tightens  locknut  on  lifting  eye. 

13.  ( ) Lowers  vehicle  to  the  ground. 

Number  of  Check  Marks 

TEST  STANDARDS: 

To  pass  this  test,  the  examinee  must  have  check  marks  in  all  13  boxes. 
The  job  must  be  finished  within  a 25-minute  time  limit. 
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3.  The  student  must  develop  a JPM  for  his  selected  task.  This  JPM  should 

Include: 

a.  The  required  test  performance— a description  of  the  action  or 
behavior  that  a learner  exhibits  during  the  test  situation.  The 
action  must  be  realistic  in  terms  of  cost,  safety,  time,  observa- 
bility, and  scorability. 

b.  Test  conditions— a description  of  what  the  learner  is  or  is  not 
given  during  the  test  situation.  This  may  include  tools,  facilities, 
environmental  conditions,  degree  of  supervision,  etc. 

c.  Test  cues— the  events  that  signal  the  initiation  or  endiriv  of  the 
task  or  some  part  of  the  task. 

d.  Test  standards— a description  of  the  index  of  acceptability  used 
to  judge  the  adequacy  of  learner  performance.  This  may  include 
the  quality,  quantity,  or  speed  of  performance. 

e.  Equipment  and  facility  requirements— any  special  equipment  or 
facilities  the  learner  will  require  to  exhibit  adequate  performance 
during  the  test. 

f.  Administration  Instructions— step-by-step  directions  for  administer- 
ing the  test. 

For  students  who  are  unable  to  develop  a complete  and  practical  0PM, 

show  them  where  they  are  inadequate  and  help  them  rewrite  the  JPM. 

(See  sample  answers  on  the  following  pages.  Sample  answer  for  the  OH-58 
Helicopter  Repairman  includes  Job  Data  Worksheets,  JPM  and  a performance 
test.) 
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Performance  Test  - Demonstrate  ability  to  inspect  landing  gear  cross 

tubes  of  an  OH-58  helicopter  safely  and  accurately. 

Test  Conditions:  Same  as  JPM 

Test  Cues:  Same  as  JPK 

Test  Standards;  Same  as  JPM 

Equipment  and  Facility  Requirements 

1.  Equipment— Oh-58  Helicopter  or  AirFrame  Training  Aid 

General  Mechanics  tool  kit 
TM  55-1520-228-20 

Plumb  line  and  bob  DA  Form  2400-13 
chalk 

Depth  gage 
Hydraulic  Jacks  (4) 

"Aircraft  on  jacks"  signs 

2.  Facility—  Aircraft  maintenance  hangar  with  cement  floor; 

appropriate  light  sets 


Scoring  Instructions 

Evaluator  will  check  the  appropriate  block  (yes/no)  on  the  attached 
performance  checklist.  Each  step  will  be.  performed  in  sequence.  Failure 
to  perform  each  step  in  sequence  and/or  failure  to  observe  safety 
precautions  will  result  in  NO  GO  rating  on  the  test. 
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JOB  DATA  WORKSHEET 
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JPM's  are  given  draft  tryouts  to  check  inter-rater  reliability 
and  to  detect  problems  with  performance,  equipment,  and  scoring 


For  process  scales  evaluate  or  check  the  steps  in  the  procedure 
Product  scales  rate  the  quality  of  the  end  items. 
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ENABLER  FEEDBACK 
MODULE  3 


Review  Sheet  for  Existing  Instruction  # 1 
For  JPM's  4 & 6 

Derision:  Reconsider  at  I I I. 3 


Reason: 


No  JPNs 


Review  Sheet  for  Existing  Instruction  2 
For.JPMs  4 4 6 

Decision:  Reconsider  at  I I I. 3 

Reason:  No  JI^s 


Review  Sheet  for  Existing  Instruction  3 
For  JPM  4 

Decision:  Accept 

Reason:  A question  should  be  raised  about  sources  of  job 

da\,a.  However,  since  the  lesson  posttest  matches  ilPMs, 
it  is  not  an  important  question. 


Review  Sheet  for  Existing  Inst’^uction  # 4 
For  JPM  6 

Decision;  The  Ordinance  Scho'J  lesson,  6051,  should  be  reconsidered 
in  III. 3. 

Reason:  There  is  n.ore  included  in  the  lesson  than  is  called  for 

in  the  task. 


Review  Sheet  for  Existing  Instruction  # 5 
For  JPM  6 

Decision:  It  should  be  reconsidered  at  III. 3 

Reason : TB-750'651  was  not  ISD'd  and  no  job  analysis  data  is 

available. 


The  trainee  must  have  a rationale  fo*'  each  decision.  If  not,  have 
him  c..mplete  one  before  discussing  tne  school  solution.  If  the 
trainee  has  different  decisions  but  sufficient  rationale  to  defend 
them,  discuss  the  above  deetSMia  .and  ret.^enales  with  him. 


2.  Existing  courses  should  be  analyzed  first.  One  of  the  primary 
reasons  for  analyzing  existing  courses  is  to  determine  if  an  ade- 
quate job  analysis  already  exists.  If  the  student  misses  this* 
discuss  the  correct  answer  with  him. 

Sample  Answer 

Analyze  documentation  first.  A review  of  source  data,  task  selection 
criteria,  resource  constraints,  source  of  task  lists  JPFfe  and  other 
data  about  whether  the  job  analysis  was  developed  through  an  ISO 
or  systems  engineering  process.  If  documentation  for  front  end 
analysis  checks  out,  the  instruction  may  only  need  update  and  verifi- 
cation without  the  expenditure  of  effort  and  resources  needed  to 
perform  a-  new  front  end  analysis. 

*** 


3.  Part  of  the  answer  to  this  question  can  be  found  in  the  student's 
data  collection  plan  developed  in  Exercise  4 of  Module  1.  The  sources 
of  data  upon  which  the  existing  courses  are  based  would  have  to  be 
reasonably  similar  to  the  sources  the  student  would  have  used.  Also, 
the  evaluation  criteria  used  to  select  tasks  trained  by  the  existing 
course  would  ha’e  to  be  reasonably  similar  to  the  criteria  the  student 
would  have  used.  In  addition,  the  student  would  need  the  JPMs 
upon  which  the  existing  courses  were  based  and  validation  data  for 
the  existing  courses.  If  the  students  reasoning  does  not  Include 
these  points,  discuss  them  with  him. 


4.  The  next  action  should  be  to  further  examine  the  job  analysis  to 
determine  how  it  was  conducted  and  whether  the  data  sources  were 
appropriate  for  the  student's  job.  (pages  221-222) 

1.  Currency  of  data  analysis 

2.  Amount  of  change  in  equipment,  doctrine,  manpower,  or  personnel 
eyetpm  c-inre  t.hp  iob  analysis  was  done 

3.  Sources  of  the  job  data 

4.  Procedures  used  in  collecting  job  data 

5.  Similarity  of  your  situation  to  the  one  on  which  job  data 
was  collected. 

Sample  Answer 

Method  of  selecting  tasks  for  training 
Criteria  for  selection 

Data  upon  which  selection  based  obtained  from  same  geographic 
locations,  skill  levels,  etc. 

Same  constraints 
Sources  of  job  data 
Review  JPMs  ff  available 
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The  development  of  new  Instruction  Is  an  expensive  and  time  consuming 
project,  if  existing  courses  are  evaluated  (according  to  the  five 
criteria  indicated)  and  found  to  be  acceptable  in  all  or  most  of  these 
criteria,  then  considerable  time,  effort  and  resources  can  be  saved. 
Analysis  of  existing  courses  also  prevents  duplication  of  effort, 
where  new  courses  are  developed  when  similar  or  suitable  courses  already 
exist. 
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ENABLER  FEEDBACK 
MODULE  4 


1.  The  workshop  participant's  responses  must  be  judged  largely  on  the 
rationale  given  for  each  setting  selection.  Based  primarily  on  the 
nature  of  the  activities  and  the  JPM  conditions*  the  following  are 
reasonable  selections  but  not  the  only  good  alternatives: 

a.  JPM  la  should  be  assigned  to  RS  or  ISS.  The  consequences  of 
Inadequate  performance  are  extremely  high,  yet  very  few  members 
of  any  unit  are  actually  required  to  perform  this  task. 

b.  JPM's  2,  3,  5,  6 should  be  assigned  to  RS  or  FOJT.  Considering 
the  high  percentage  of  members  who  perform  this  function  RS 
would  be  the  most  cost  effective  method.  ISS  would  lead  to 
much  duplication  of  effort* -If  a large  number  of  schools  each 
conducted  a course.  Portions  of  3,  5,  and  6 could  be  provided 
by  JPAs  to  be  used  as  a guide,  while  performing  the  task.  If 
graduates  are  not  required  to  perform  the  task  soon  after 
training  then  FOJT  may  be  more  appropriate  than  RS. 

c.  JPHs  7,  8,  9,  10  should  be  assigned  to  RS  or  FOJT,  depending 
upon  how  soon  after  assignment  the  graduate  must  perform  the 
tasks.  Although  consequences  of  inadequate  performance  are 
moderate,  task  difficulty  Is  reii.-i.ely  high  on  RS  or  FOJT 
settings  would  provide  opportunities  for  students  to  receive 
feedback.  Portions  of  7,  8,  9,  10  could  be  assigned  to  STEPS. 


The  student  must  have  data  from  which  can  be  determined  who  needs  ' 
the  training  and  when  the  training  will  be  required.  The  data  should 
Include  much  of  what  is  covered  In  the  summary  data  from  exercise 
one  above.  It  does  not  have  to  be  identical  In  format  on  criteria 
selected.  He  also  must  make  assumptions  about  resource  availability. 


Sample  Answer 

(See  sample  answers  In  Module  2,  exercise  3) 
For  JPM  # 1 

JOB:  Helicopter  Mechanic 
DUTY:  Performing  Inspection 
TASK:  Inspect  landing  crosstubes 
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Additional  data  needed: 


a.  Perfomied  at  skill  level  1-4 

b.  Medium  delay  tolerance 

c.  Follows  set  procedure 

d.  Little  physical  skill  required 

e.  Requires  supervision  or  following  Inspection 

f.  Resource  constraints—avallablllty  of  helicopters  for 
training 

g.  New  assignees  required  to  perform  task 

h.  Low  decay  rate 

For  JPM  I 2 

JOB:  Flight  Operations  Coordinator 
DUTY:  Processing  flight  plans 

TASK:  Evaluate  DO  Form  175  for  conpletei^ss  and  accuracy 
Additional  data  needed: 

a.  Performed  at  skill  level  2 or  3 

b.  Low  delay  tolerance 

c.  Follows  set  procedure 

d.  Conditions  not  a factor 

e.  Little  physical  skll'  required 

f.  Doer:  not  require  close  supervision 

g.  No  critical  resource  o)nstra1nts 

h.  New  assignees  not  required  to  perform  task 
1.  High  decay  rate 


3.  Each  instructional  setting  and  rationale  must  be  consistent  with 
the  guidelines  provided  In  Block  1.5,  pages  244-262.  If  the 
settings  are  Inconsistent  discuss  other  options  with  the  student. 

The  Enabler  has  the  final  say  In  selecting  settings.  If  the  JPA 
Is  the  most  appropriate  setting  for  all  tasks,  have  the  student 
write  arrather  task  statement  that  Is  appropriate  for  any  o^r 
setting  In  order  to  do  exercises  In  other  modules. 

Sample  Answer 

For  JPM  # 2 

Nominate  for  STEP  and  JPA  based  on  rationale  In  a,  c,  f,  g,  h 
For  JPM  # 1 

Nominate  for  RS  and  JPA  based  on  rationale  In  a,  e,  f,  g,  h 

*** 
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enabler  feedback 

MODULE  5 


1.  The  test  items  should  adequately  measure  the  capability  to  be  learned. 
Student  test  items  should  follow  the  guidelines  noted  in  the  Phase  li 
Manual  on  pages  39-46.  If  student  answers  do  not  follow  recommended 
practice  refer  them  to  these  pages.  The  importance  of  high  fidelity 
between  TLOs  and  test  items  should  be  noted. 

The  test  items  should  be  similar  to  these: 

a.  Your  starter  failed  to  crank  and  you  have  performed  a starter 
voltage  test.  Use  tm  9-2320-218-20.  The  voltmeter  looks  like 
this  (or  reads  19.0).  What  should  you  do  next? 


If  the  voltmeter  reads  17.0,  what  should  you  do  next? 


b.  Your  starter  failed  xo  crank,  you  performed  test  2,  and  then  test  3. 
The  reading  is  more  than  0.1  volts.  What  should  you  do  next? 


Analysing  test  results.  What  do  che  following  mean? 

1.  Starter  voltage  test:  /show  \ 

/ readings) 

\on  LVCT/ 


2.  Starter  amperage  test:  /Show  \ 

( readings 
\on  LVCT/ 


3.  Battery  ground  cable  test:  /Show  \ 

/ reading^ 
\on  LVCT/ 
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4.  Battery  cable  test: 


/show  \ 

/ readings) 


Depending  on  t^ 
Important  thin 
Items  require 


tests  selected,  the  test  items  will  vary.  The 
j that  they  include  the  use  of  the  TM,  and  that  the 
erpretation  of  the  results  of  the  tests. 


2.  Each  TLO  should  match  its  JPM  as  closely  as  the  learning  environment 
allows.  The  actions,  conditions  and  standards  for  each  TLO  should  match 
the  actions,  conditions  and  standards  of  the  JPM  as  closely  as  the 
learning  environment  allows.  Blocks  1-4  of  the  LOAW  should  be  complete. 
Look  at  the  JPM  from  Module  2. 

(See  Sample  Answer  pages  3-7.) 

3.  Each  TLO  must  be  classified  into  the  appropriate  learning  category: 
Physical  skill,  mental  skill,  information  or  attitudes.  (Pages  6-16 
of  the  Phase  II  Manual.)  Tell  the  student  which  ILOs  he  should  use 
for  Exercise  4.  Assign  one  LO  (preferably  in  the  mental  skill  or 
physical  skill  category)  for  Exercise  4.  (See  block  6 on  the  LOAW.) 

(See  Sample  Answer  pages  3-7.) 

4.  The  Enabler  should  have  assigned  a TLO  for  the  analysis,  preferably  in 

mental  skills  or  physical  skills  category.  The  product  should  be 
o-^'-lete  in  accordance  with  guidelines  given  for  the  particular  learning 
agory.  LSs  should  have  been  listed  in  block  10  of  the  LOAW. 

Mental  skills:  pages  19-23 
Information:  pages  23-26 

Physical  skins:  pages  26-28 

Attitudes:  pages  28-30 


(See  Sample  Answer  page  8.) 


5.  The  student  entry  behavior  assumptions  should  be  apprroriate  to  the 
particular  workshop  participant's  tasks  and  probable  student  population. 
Entry  behavior  assumptions  are  discussed  on  page  18  of  the  Phase  II 
Manual . 

6.  Check  the  test  items  to  be  sure  all  TLOs  are  tested.  Since  TLOs  include 
LOs  and  LSs,  a separate  test  item  is  not  always  necessary  for  the  TLO. 

For  LOs  that  include  a number  of  LSs-,  LSs ■may^be''tested‘a$T;part  of  the, 

LO  test.  Test  items  that  test  entry  behavior  chould  be  included.  Trans- 
lating learning  objectives  into  test  items  is  discussed  on  pages  39-46 


LEARNING  OBJECTIVE  AN ALYSIS  WORKSHEET 


Sample  Answer  (2  & 3) 
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W1th1n>course  tests  snould  be  used  when  they  are  needed  to  test  those 
steps  Involved  in  learning  a task,  and  when  standards  need  to  be 
raised  during  the  learning  process.  They  are  also  progress  checks  for 
the  student.  If  none  of  the  above  are  useful,  then  you  may  not  use 
them. 


8.  False  positives  and  false  negatives  misclassify  students  Into  the 
wrong  categories.  If  a person  cannot  perform  the  task  but  he  passes 
the  test,  he  Is  called  a false  positive.  If  he  can  perform  the  task 
but  fall^  the  test,  he  Is  called  a false  negative.  Tests  should  be 
designed  so  that  the  fewest  possible  misclassificatlons  are  made. 

This  Information  Is  helpful  in  setting  cut-off  scores. 

9.  High  physical  fidelity  means  that  the  test  "looks"  very  much  like  the 
job. 
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ENABLER  FEEDBACK 
MODULE  6 


1.  Entry  behavior  assumptions  and  test  Items: 

Assumption  1 was  not  tested.  The  trainees  should  note-the  deficiency. 

The  results  neither  confirm  nor  deny  the  assumption.  It  Is  possible 
that  an  Individual's  reading  level  is  a part  of  his  records;  If  not, 
a short  test  of  reading  the  specific  TM  should  be  Included. 

The  learning  analysis  is  estended  to  te.<>t  assumption  1:  Trainees  can 
read  well  enough  to  use  TMs.  Since  trainees  are  required  to  read  TMs  as 
part  of  the  training,  assessing  their  reading  on  an  actual  TM  has  high 
predictive  validity. 

Assumption  2 was  probably  insufficiently  tested.  What  trainees  say  they 
will  do  Is  not  necessarily  what  they  do  on  the  job.  Either  a performance- 
oriented  test  that  Includes  some  check  of  the  safety  rules  should  be 
developed,  or  safety  precautions  specific  to  this  equipment  should  be 
Included  In  the  Instruction. 

If  trainees  do  not  notice  the  problems  with  assumptions  1 and  2,  point 
them  out. 

Assumption  3 was  confirmed.  The  trainee  may  delete  it  from  the  entry 
test.  If  he  does  not,  suggest  it. 

Assumption  4 was  not  confirmed.  Correct  connections  and  assembly  of 
support  equipment  siicild  be  included  in  the  Instruction. 

Extension  of  learning  analysis: 


A. 1.1. 1.4 
Use  TM 

9-4910-456-14 


B.I.I.; 


Use  TM 


A. 1.1. 1.4.1 


B. 1.1. 2. 4.1 


Read  TM 


Read  TM 


61 


TW-EF6-2 


2.  Although  the  LOs  are  not  dependent,  the  sequence  In  which  they  are 
learned  should  follow  a logical  order.  The  student  should  understand 
this  logical  relationship;  for  example*  safety  rules  should  be  learned 
before  assembling  equipment.  The  sequencing  must  Include  simultaneous 
coverage  of  the  Identical  LOs  first. 


a.  A. 1.1. 1.4.1  and  B. 1.1. 2. 4.1  (These  LOs  were  added  to  test  entry 
behavior  assimiptlon  one.  The  LOs  are  to  read  the  TM.) 


A.l. 1.1.1  and  B. 1.1. 2.1 
A.1.1.1.2  and  B. 1.1. 2. 2 
A.1.1.1.3  and  B.1.1.2.3 


follow  safety  rules) 
^Identify  parts) 
assemble  equipment) 


b.  A.l. 1.1.4  and  B.l. 1.2.4  (use  TM) 


c.  A. 1.1.1  and  B.l. 1.2  (operate  equipment) 


d.  Either  A. 1.1,  A.l,  and  A In  that  order  or  B.l. 1.1. 1.1,  B.l. 1.1.1, 
B.l. 1.1,  B.1.1,  B.l  and  B In  that  order,  followed  by  the  other. 


3.  The  entry  test  Items  should  match  the  entr'  behaviors. 

a.  Test  development  was  covered  In  I 1.2.  The  pages  by  learning  category 
ar-e: 


/iental  skills 
Information 
Physical  skills 
Attitudes 


pages  39>42 
pages  42-44 
pages  44-45 
pages  45-46 


(Sample  answer  Is  on  pages  3 and  4.) 


b.  Standards  and  scoring  are  also  covered  in  II. 2 on  pages  47-56. 

c.  Protest  use  Is  covered  in  ii.3,  pages  73-75.  The  student’s  response 
to  this  question  should  Include  the  use  of  the  pretest  for  evaluating 
the  Instruction  and  whether  to  use  It  for  placement  within  the  course 
or  to  exempt  the  Instruction. 
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Sample  Answer  (3a)  (con't) 

2.  What  do  the  following  symbols  on  low  altitude  chart  represent? 

a.  

b.  A 

c-O  



3^^ ▲ 


3.  Using  DoD  FLIP  General  Planning  Manual  ard  Information  provided* 
complete  following  entries  on  DD  Fonn  175; 

Complete  blocks  1 through  10 

(form  omitted  here  to  conserve  space) 

4.  Using  data  provided  and  the  CPU-26  computer,  perform  the  following 
calculations: 

a . Rate  90K 

Time  2 + 30  hours 

Distance  flown  

b.  Time  2 + 10 
Distance  flown  180  NM 

Rate  

c.  Magnetic  course  120° 

Winds  010/10 

True  airspeed  lOOK 

Ground  speed  

Wind  drift  correction 


**<r 
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4.  The  students  should  already  have  at  least  enough  Iteis  for  either  a 
pretest  or  a posttest.  Hhere  possible  they  should  rewrite  alternate 
forms  for  the  Items.  The  Items  should  be  as  performance  oriented  as  Is 
reasonable.  If  the  Items  are  not  performance  oriented,  ask  for  the 
rationale. 

(Sample  Answer  Is  on  paces  6-8.) 


5.  Check  the  learning  analysis  for  the  se<^enc1ng.  Be  sure  prerequisite 
objectives  precede  the  dependent  ones.  Studeits  bhould  have  a rationale 
for  sequencing  that -Is  not  consistent  with  accepted  netho<k)1ogy. 

(Smnple  Answer  Is  on  page  g.) 


6.  It  Is  unnecessary  to  sequence  Independent  objectives  at  this  point,  since 
such  objectives  can  be  arranged  In  any  sequence  without  loss  of  leamli^. 
This  Is  discussed  on  pages  83  and  90  of  the  Phase  II  Manual. 


7.  A pretest  matches  the  posttest.  It  can  be  used  to  place  a student  within 
a course  or  to  exempt  a course.  An  entry  test  measures  what  the  learner 
must  be^able  to  do  before  he  starts  the  Instruction. 


8.  Dependent  learning  objectives  must  be  sequenced  based  on  thelv  relation 
ship.  Indepenlent  objectives  can  be  sequenced  In  any  logical  way. 


Sample  Answer  (4) 


Pretest: 

1.  Using  Panel  C,  Chart  L-18  and  straight  edge,  measure  distance  from 
Montgomery  (MGM)  to  Fort  Campbell  (HOP)  using  V-20  airway. 

2.  Which  of  the  following  are  optional  entrit'  on  DD,  Form  175  (refer  to 
FLIP  General  Planning  Manual,  if  necessary)? 

a.  Transponder  Code  (Block  7) 

b.  Briefing  Void  Time  (Block  31) 

c.  Type  of  Flight  (Block  Pi 

d.  Fuel  on  Board  (Block  17) 


Situation  1:  You  have  received  a flight  plan  for  s flight  from  Cairns 
Army  Airfield  to  Oar.nelly  Field,  Montgomery  using  the  route  of  flight 
as  follows.  Flight  altitude  is  4,000  feet. 

Leg  1 - CAAF  to  Skippervllle  In  section  using  Sklppervllle 
Standard  Instrument  Departure  fSID)  #2. 

Leg  2 - Sklppervllle  to  Banks  or.  V-7  airway. 

Leg  3 - Banks  to  Shady  Grove  Intersection  - V-7A  airway. 

Leg  4 - Shady  Grove  direct  MGM  VOR 

Figure  1-- 


8or\k.s 


10  5 


20 


I 


SCALE  (in  Nautical  Miles) 


0 


T 
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Sample  Answer  (4)  (con't) 


3.  The  distance  from  CAAF  to  Banks  Intersection  Is: 

a.  16  nautical  miles 

b.  16  statute  miles 

c.  34  nautical  miles 

d.  34  statute  miles 


Situation  2:  Pressure  altitude  - 4,000‘ 

Air  temperature  at  4,000’  - +15°c 
Calibrated  airspeed  - 90K 

Using  the  CPU-26  computer  provided  and  the  information  provided  1n 
Situations  1 and  2,  true  airspeed  for  this  flight  would  be: 

a.  102  knots 

b.  92  knots 

c.  102  miles  per  hour 

d.  92  miles  per  hour 


Situation  3:  Use  Information  contained  In  Situations  1 and  2 and 
the  Information  listed  belov;. 

Wind  direction  (magnetic)  - 030°  (at  4,000'  altitude) 

Wind  speed  - lOK  (at  4,000'  altitude) 

Average  ground  speed  between  Skippervllle  In^..rsect1on  and  Shady  Grove 
Intersection  is: 

a.  74K 

b.  84K 

c.  74  mph 

d.  84  mph 


6.  Based  on  the  ground  speed  determined  In  Item  5,  your  estimated  time 
enroute  from  CAAF  to  M6M  should  be: 

a.  1 + 45  minutes 

b.  1 + 50  minutes 

c.  1 + 31  minutes 

d.  1 + 41  minutes 
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Sample  Answer  (4)  (con't) 


7.  Compare  the  ETA's  stated  below  and  identify  the  correct  action. 

Pilot's  ETA  (CAAF  to  MGM):  1 + 39 
Your  calculated  ETA  (CAAF  to  MGM:  1 + 41 

Select  the  appropriate  response  for  action  required. 

a.  Approve  the  flight  plan 

b.  Disapprove  the  flight  plan 

c.  Re-calculate  ETA 

d.  Direct  pilot  to  change  route  of  flight 
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ENABLER  FEEDBACK 
MODULE  7 


a.  The  correct  subcategories  are: 

LO-1  "Operate  LVCT"  is  subcategory  9,  positioning  movement 
and  recalling  procedures.  The  action  verb  "operate"  is  in 
the  list.  It  includes  the  cognitive  and  motor  aspects  of 
equipment  set-up,  principally  motor  since  the  cognitive  part 
is  simple  and  illustra  2d  in  the  manual. 

LO-2  "Select  test  for  given  symptoms  or  test  results"  is 
subcategory  1,  rule  learning  and  using.  The  action  verb  is 
"select".  It  is  definitely  a case  of  choosing  a course  of 
action  based  on  applying  known  rules. 

LO-3-1  "Perform  the  battery  positive  terminal  test"  is  sub- 
category 2,  classifying  as  are  the  LO-3-2  and  LO-3-3,  because 
the  motor  part  is  relatively  simple  but  the  important  part 
is  to  determine  if  the  battery  is  serviceable  or  needs  replacing. 

b.  See  attached  learning  objective  analysis  worksheet  (reverse 
side  page  2). 


The  classification  of  each  learning  objective  should  be  in  accordance 
with  the  outline  on  pages  12-16,  Table  III. 6 Description  of  Training 
Task  Categories.  Familiarize  yourself  with  this  table  in  advance 
so  you  can  use  it  quickly  and  effectively.  Students  who  submit 
incorrectly  classified  learning  objectives  should  be  shown  which  are 
wrong,  and  how  to  correct  them.  You  can  review  pages  12-16  with  him 
if  he  has  problems.  (See  Sample  Answer  pages  3-7.) 


It  is  unnecessary  to  include  all  of  the  guidelines  listed  for  each 
category  of  learning.  It  is  more  appropriate  to  select  only  those 
which  are  essential  to  mastery  of  the  learning  objectives. 

The  selection  of  appropriate  guidelines  requires  judgement.  Insure 
that  each  of  the  guidelines  listed  is  both  appropriate  and  necessary. 
Then  quickly  scan  the  guidelines  yourself  to  assure  that  no  important 
ones  have  been  omitted.  Be  particularly  alert  for  guideline  sequences 
which  omit  the  four  general  guidelines: 

1.  Inform  the  learner  of  the  objectives. 

2.  Provide  for  active  practice. 

3.  Provide  guidance  and  prompts. 

4.  Provide  feedback. 


Learning  Activity 

State  the  objectives 

Demonstrate  the  procedure 
visually 

Use  the  diagrams  in  the  TM 
or  print  them.  Inform  the 
trainee  that  the  TM  is  a 
job  aid  he  can  always  use 

Provide  practice,  especially 
reading  the  voltmeter 

Provide  visual  display  of 
correctly  set  up  LVCT  to 
batteries 

Provide  description  of  the 
task  in  relationship  to 
other  similar  ones 


I 

4 


10 


14 


20 


COMPLEXITY  CRITERIA 
Oirficult  Motor  Aeu. 


Smooth  Motor  Porformane*  at  and  of 
Training  

STIMULUS  CRITERIA 
VltualFerm 
AlphatHj  marie. 


Pioiorlal, Plana  . ..  ■ .. 
Una  Conamiction,  Plana. 
Oblact,  Solid. 


PuR  vlauai  SKvIronman;. 
Viauai  Moaawant 

MU 


LimlMd. 

Fun 


VMual  Qpactrum 
Blaek  and  Whlta* 
Qray  tcala 
Color 

SeWa 

Exact  Seala 

Audio 


Voloo  Sound  Ranga- 
FullSoundRanga— 


AntiiantSoundi. 
Othar 

TaetlloCuoa. 


Intamal  Stbnuluc  Motion  Cuaa. 
SMomal  SiimuliM  Motion  Cuaa- 


Pina  mommant  manlpulativa  Aeta. 


Broad  Movarnam  manipulathia  A«r_ 


TRAININO  SETTINO  CRITERIA 

IndIvidMai  Trainaa  or  taam  training  at  a 
Fixad  Location  


IndMdual  Trakiaat  with  aimuitanaoM 
inatruoMon  or  many  looationa  ~_ 


IndMdual  Trainaa  or  taara  trainino  with 
Indapandant.lnatftietlon  at  any 
Location — 


individual  TrWnaa  on-tha-iob_ 
SmallOioup 


Larga  Oroup  at  lingia  Location . 
Taam  Sotting  ■—  — .... 


ADMINISTRATIVE  CRITERIA 
Sha  Of  Counawiara  Dtvaiopmant 
Local—.  . ■ 


CantraL. 


MagnHuda  of  AoguWtIon  Coat 
Low 


aat— a. 
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If  a list  of  guidelines  contains  inappropriate  entries,  explain 
to  the  student  why  each  is  inappropriate.  If  critical  guidelines 
are  missing,  require  students  to  make  additions  to  their  lists. 
(See  Sample  Answer  page  9) 


4.  The  student  must  match  each  specific  learning  guideline  with  a 
specific  learning  activity.  Again,  a good  bit  of  judgement  is 
required.  For  students  who  are  unable  to  develop  an  acceptable 
learning  activity  for  each  learning  guideline,  review  pages  24-26 
and  discuss  learning  activities  for  his  objectives.  (See  Sample 
Answer  page  10) 


5.  Natural  feedback  is  something  that  occurs  as  a result  of  performing 
an  action  that  lets  the  perfonrier  know  if  it  is  right  or  wrong,  good 
or  bad,  finished  or  not  finished,  etc. 

Artificial  feedback  is  something  that  is  made  to  occur  to  let  the 
performer  know  if  he  is  right  or  wrong,  good  or  bad,  finished  or 
not  finished,  etc. 

A learning  activity  is  the  specific  behavior  a student  perforas 
during  a particular  episode  of  learning. 
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Sample  Answer  (3) 


The  guidelines  chosen  are: 


SUB-CATEGORY  1:  RULE  LEARNING  AND  USING 


Guideline  No. 


1.  At  the  beginning  of  the  training,  the  instructor  or  the  materials 

should  clearly  inform  the  trainee  of  the  learning  objectives; 
that  is*,  what  the  trainee  is  expected  to  be  =hle  to  do  by  the 
completion  of  training.  Re’ate  the  objectives  and  learning 
activities  to  operational  tasks  which  the  trainee  must  perform 
in  future  real-world  assignments. 


4.  Present  examples  of  when  the  rule  anplios,  and  when  it  doesn't. 


5. 


9. 


Provide  opportunities  to  apply  the  rule  in  a variety  of  new 
situations  in  which  the  learner  has  not  previously  been  trained 
to  apply  the  rule.  During  practice,  practical  applications  and 
practice  tests  provide  the  student  with  iirwedi ate  knowledge  of 
results  about  his  correct  and  inco>^rect  answers.  Provide  re- 
wards for  correct  application  of  the  rule. 


Relate  the  rules  to  be  learned  to  operational  tasks  which  the 
trainee  must  perform  in  future  real-world  assignments. 


Le»rning 
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f:  9 


J I lifting  Activity 

State  objectives  ve*"bany 
and  in  w»-it.1ng 

; Explain  rule 

; Denonstrate  use  of  rule  to 
; solve  problem  using  computer 
and  diagrams 

! Provide  practice  in  applying 
i rule  to  solving  problems  to 
! be  of  increasing  complexity 

; Relate  rules  to  real  world 
tasks 


COMPLEXITY  CRITERIA 

Uitficuh  MotOf  Act! 

Smoor-'  Motor  Ptrformanct  at  r id  of 
Trainioj - - — 

STIMULUS  CRITERIA 

I VIhnI  Fotm 

1 AlpiMTiurrwric. 

Pictorial,  

Lint  Conttructioo,  PliAr  

Object,  Solid— 

PuU  viniai  •tttfiromnom — 

Vituil  Mc-^.tw<rt 

Still — 

Limited  

Full — 

ViMi  Ipcctrutn 

Olock  end  White-  ■■ 

Grey  Icaic — 

Coi'of  ' — — — 

Se«l« 

Exact  Scaia 

Audio 

Voioa  Sound  Ranga 

Full  Sound  Ranga- 

Atr<>iant  Sound*— 

Othc. 

TactllaCoa*- 

Intarnal  Stimulu*  Motion  Cuat 

Extamai  StimuM  faction  Cua« 

Pina  mewntaw  inonipulatiM  Act* — 
Broad  Mowmant  manipulativa  Actt_ 

TRAINING  setting  CRITERIA 

lirdividuai  Trstnat  cr  team  training  et  » 
Fixad  Location  — 

iridivtduM  Trainaa*  Wib  timultanaou* 

in*tructl«st  or  many  iocatton* 

Indhrldwai  TrNnaa  cr  tMT.  training  v,  h 
Indapartdant.lnuructlon  at  any 

Location 

ind'ivio>!ai  Tralnaa  on-thn-job_____ 

Small  Group 

Larga  Group  at  Blnipa  U-cat  icn 

Taam  Sattbig 

ADMINISTRATIVE  CRITERIA 
Sita  o(  Courtwara  Oavdopmom 

Local — 

Central 

Magnltuda  of  Acquiotlon  Co*t 

Low — 

High 


Madia  Salsctio'*  ar  d Rationale- 


ENABLER  FEEDBACK 


MODULE  8 


1.  See  pages  2 and  3. 


2.  Media  selection  includes  a series  of  interdependent  steps.  The  only 

way  to  assure  that  a student  has  made  the  correct  choice  is  tc  examine 

his  work  in  light  of  th’s  sequence  of  steps: 

a.  The  student  begins  with  a sequence  of  learning  activities  developed 
in  Block  III.l.  These  are  entered  in  ^he  learning  activity  column 
on  the  left-hand  side  of  the  worksheet.  Has  the  student  listed 
then  properly? 

b.  A stimulus  criterion  is  selected  for  each  learning  activity.  Only 
those  stimulus  criteria  which  are  absolutely  necessary  should  be 
marked.  This  will  yield  a selection  which  is  the  least  expensive 
media  available  to  do  the  job.  Has  the  student  selected  the  mini- 
mum stimulus  criteria? 

c.  The  student  turns  to  Appendix  B,  pages  174-184,  and  locates  the 
matrix  appropriate  to  the  sub-category  into  which  his  learning 
activities  fall.  Has  he  selected  the  correct  matrix? 

K<;  next  marks  the  delivery  system  most  appropriate  to  his  stimulus 
criteria,  training  setting  criteria,  and  administrative  criteria. 

Are  the  student's  selections  consistent  with  the  three  criteria? 

d.  The  student  now  has  a media  pool  consisting  of  several  media  choices. 
His  final  decision  is  to  select  the  optimuiii  media  mix  by  eliminating 
impractical  approaches  occording  to  the  11  criteria  listed  on  pages 
120-121.  Has  he  dorte  so?  Is  his  rationale  in  keeping  with  the 
criteria?  Each  incorrect  stage  must,  be  corrected  before  the  next 
stage  v.an  oe  judged. 

(See  Sample  Answer  page  n.) 


Student  responses  should  include  dicsussion  of  all  10  factors  (A-J) 
in  line  with  their  objecti.-es,  learning  activities,  and  delivery 
system  (media  selection)  as  specified  under  "Determine  Management 
Guidelines,"  pages  124-137,  Block  III. 2.  Student  answers  of  10  factors 

enough  for  someone  else  to  understand  and  follow  the 
rationale  for  the  selected  management  plan.  Students  who  are  unable  ^-o 
generate  a meaningful  response  for  any  factor  should  reread  the  oaaes 
corresponding  to  ^hat  factor  and  try  again.  ^ 

(See  Sample  Answer  pages  5-8.) 


operate  LVCT 


Act.vity 

f 

I state  the  objectives. 

Demonstrate  the  procedure 
visually. 

Use  the  diagrams  in  the  TM 
or  print  them.  Inform  the 
trainee  that  the  TM  is  a 
job  aid  he  can  always  use. 

Provide  practice  especially 
in  reading  the  voltmeter 

Provide  visual  display  of 
correct! V set  up  LVCT  to 
batteri'.' . 

Provide  description  of  the 
task  in  relationship  to 
i other  similar  ones. 


COMPLeXirV  CRITERIA 

Ditficult  Motor  Acts 

Smooth  Mciot  PeiformBiice  at  end  of 
Traimno 


SI  IMULUS  CFtITERIA 
Viiual  Form 

Alphanumeric 

Pictorial,  Plane 

Line  Construction,  Plane 

Object,  Solid 

Full  visual  environment 

Visual  Movement 

Still — — 

Limited 

Full 

Vuuel  Spectrum 

Black  end  White 

Grav  S ele 

Color- — — 

Scele 

Exact  Scale 

Audio 

Voice  Sound  Range-  — 

Full  Sound  Range 

Ambient  Sounds  — - 

Other 

Tactile  Cues — - — 

Internel  Stimulus  Motion  Cues 

External  Stimulus  Motion  Cues 

Fine  ntovement  manipulative  Acts — 
Broad  Movement  menipulative  Acts_ 

TRAINING  SETTING  CRITFRIA 

Individual  Trainee  or  team  training  at  a 

Fixed  Looetion 

Individual  Trainees  with  simultaneous 

instruction  or  many  locetions 

Indivxjut!  Trainee  or  team  training  with 
Independent  Instruction  at  any 

Location 

Individual  Trainee  on-the-job 

Smelt  Group 

Large  Group  at  Single  Location 

Taam  Setting 

ADMINISTRATIVE  CRITERIA 
Site  of  CoursewB'e  Development 

Local 

Cent  rel 

Magnitude  of  Acquisitior.  Cost 

Low 

High 


Logic  trainer  with 
tutor. 


Lab  carrel  with 
equipment  and  linear 
instructional  materials. 


■ 


Media  Selection  and  Rationale' 


Lab  carrel  with  equipment 
and  linear  instructional 
materials. 

This  is  a relatively 
simple  task  on  readily 
available  equipment. 

If  developed  for  this 
purpose  it  can  also 
be  used  FC'f. 
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Ui  rcc t lOnA : 

To  choose  A delivery 
syst«iKi 

1.  Place  a 'J'  (ll')hc 
pencil)  in  boxes 
representing  criteria 
(rows)  that  must  be  met. 

2.  Select  the  delivery 
systems  (columns)  that 
have  an  *'X'*  in  each  row 
dcsl*jnaicd  by  a "'J*  • 
These  arc  the  csndtdati. 
delivery  systems. 


Alternative  instructional  Deliv»'iv  ‘.y^tofp* 


Criteria  for 
Select*  aq 
Inst  ruct 1 one  1 
Oe 1 1 very 
Sysfcps 


coBpli X ity  Criteria 

•difficult  Motor  Actn 

•S-uftth  Motor  Performance  at 
fed  of  Ti  A 1 j . 9 

:»\ir«olus  Criteria 

• Visu.  1 1 oi m 

A)plia-Numrric 
fi**torial«  Plane 
Solid 

.*t  . * 1 

r*j  1 l “eve-went 

• Audio 

Voict  SounJ  Kanqe 
fu 1 * Sour  d hanoe 
Amblvit  Sounds 

• Other 

Tactile  Cues 

Ir.toroal  stimulu'i  Motion  Cues 


Delivery  Approaches 
Permittinq  the  Application 
of  All  bcarninq  o'uidelincs 
and  Alqorithm 


Delivery  Apprcaehi  s NOT 
Pcrp.ittxnq  Cor\|)lete 
Application  of  :.c.irh;nq 
Ou  ide  1 1 he*,  and  • A I q.'' r 1 1 hn 


X 

X 

X 

X 

X 

X 

— 

X 

X 

r 

X 

X 

-Jd 

X 

X 

X 

X 

X 

X 

X 

X 

X. 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

• 

X 

X 

X 

X 

X 

X 

X 

X 

X 

ij 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

4— 

XXX 


"idinii.q  SelMhC  Criteria 

•/ndividual  Trainee  At  rjxed  Location 

•rnd^vidua!  Trainee  with  Independent 
Instruction  Any  Location 

• Sva  I I Croup 

• Larqt.  s*ro*  at  Single  Location 

• Tran  1 1 nq 

Administr.'tivr  criteria 

^i*'itc  of  Cour-.ewaro  and  Special  Hard- 
w.^ro  Devc  lopnent 

Loca  I 
< ent  ra  I 

• Kaqnitnde  of  A<  j'li  a 1 1 ' oti  Cost 

Low 
H 1 <i  h 


X X X X K X XXX 

X X 
X 

X X X X 


X X X X X X 


1 

1 - Guide 

Leirning  Category 

Rule  Learning  and  Using 

Learning  Activity 

I State  objectives  verbally  1 

1 and  in  writina. 

Explain  rule. 

Demonstrate  use  of  rule  to 
solve  problem  using  compu- 
ter and  diagrams. 

Provide  practice  in 
applying  rule  to  solving 
problems.  Problems  to  bo 
of  increasing  complexity. 


Relate  rules  to  real  world 
tasks. 
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COMPLEXITY  CRITERIA 

Difficult  Motor  ■ — — 

Smooth  Motor  Perfcrmar  at  and  of 
Trelning- 

STIMULUS  CRITERIA 
VImmI  Form 

Alphanumeric 

Pittorial.  Plane 

LbM  Conttruction,  Plm~ 

Obiect,  Solid-  — 

FuN  viMial  environment. 

VlwW  Movantent 

Still 

Limhad — 

FuH. - 

Vliual  Spectrum 

Stack  and  Vttiite — 

Gray  Seale— 

Color — 

Seek 

Exact  Seale  — — 

Audio 

Voiea  L^und  Range. — 

Full  Sound  Range  . 

Ankiant  Sounda^ 

Other 

Tactile  Cuei— — 

Internal  Stimulu*  Motion  Cuot 

external  Stimuiut  Motion  Ouei 

Fine  movemam  manipulative  Acte 

Broad  Movement  manipulative  Acti_ 

TRAINING  SETTING  CRITERIA 

Individual  Tralnao  or  team  training  at  a 
Fixed  LocePOf  I ■ 

Individual  Trainaaawhh  limultanaout 

IneirtMion  or  many  location* 

iiidivkiuai  TrainH  or  team  training  with 
Indapiwdant.lnetructlon  at  any 

Location  — ■ ■ 

Individual  Trainee  on.the.iob 
Smell  Group 

Large  Group  etShtgie  Location 

Team  SettPig  — ■ — - 

ADMINISTRATIVE  CRITERIA 
Site  of  OKTirvara  Oavalopmam 

Local 

CentiaL 

Magnttuda  of  Aoquidtion  Covt 
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Media  Pool 


Mtdia  Selection  and  Rationale 


Programmed  Text  (linear) 

Printed  matter 
Self-scoring  exercises 
Workbook 

Traditional  Classroom 

1.  Combination  satisfies 
all  criteria 

2.  Conducive  *o  self- 
pacing 

3.  Minimize  instructor 
requirement 

4.  Capitalize  on  use  of 
existing  materials 


High 
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Sample  Answer  (3) 


INSTRUCTIONAL  MANAGEMENT  PLAN:  Flight  Operations  Specialist  Course 
I . Mode  of  Instruction: 

Group,  self-paced.  Refer  to  Program  of  Instruction  and  Whekly  Training 
Schedule.  Classes  designated  as  mandatory  for  group  meetings  are  annotated 
with  double  asterisk  (**). 


II.  Course  Management: 

1.  Students-- 

a.  Ent*"y  - Entry  into  a phase  of  training  will  be  determined  by  completion 
of  the  appropriate  programmed  texts  and  completion  of  requi red  entry 
and  end-of-phase  tests.  Entry  into  advanced  phases  will  be  granted 
only  upon  attainment  of  a minimum  score  of  85%. or  higher  on  the  enc'.-of- 
phase  test.  Students  scoring  75%  or  higher  on  a phase  or  block  pretest 
may  be  excused  from  formal  instruction  but  eve'^ybody  must  score  at 
least  85%  on  posttest  or  take  remedial  instruction. 

b.  Early  completion  - Student  is  considered  to  have  completed  the  course 
upon  completion  of  required  instruction  and  achieving  test  scores 
outlined  above  and  in  the  course  syllabus.  Academic  records  and 
certification  of  completion  of  course  requirements  will  be  forwarded 
to  Director,  Resident  Training  Management,  for  students  completing  the 
course  ahead  of  scheduled  class  completion  data.  Such  individuals 
will  be  reported  to  AC  for  reassignment  within  72  hours. 

c.  Recycling  (academic)  - Students  failing  to  achieve  minimum  standards 
for  advancement  will  be  recycled  through  the  phase  at  which  failure 
occurred.  Academic  and  personnel  records  of  individuals  twice  failing 
a particular  phase  will  be  foiTTfarded  to  the  academic  review  board  for 
consideration  for  relief  from  the  course.  Academic  records  will 
reflect  record  of  counselling  sessions  required  by  course  syllabus 
and  instructor's  guide. 

d.  Program  incomplete  - Students  who  fail  to  complete  the  course  (except 
as  outlined  in  (c)  above)  due  to  illness  or  other  emergency  will  re- 
enter the  course  at  the  beginning  of  the  first  phase  after  the  last 
one  successfully  completed,  providing  he  re-enters  within  30  days. 

For  absences  greater  than  30  days,  the  individual  must  re-enter  as  a 
new  student  and  may  advance  as  prescribed  in  1(a). 

8.  Program  completion  - At  the  end  of  the  seventh  week  of  instruction, 
a roster  of  individuals  who  will  graduate  on  schedule  will  be  forwarded 
to  AG  for  publication  of  assignment  instructions.  Changes  to  this 
roster  may  not  be  made  up  after  the  tenth  week  except  for  exceptions 
outlined  in  1(c)  and  1(d)  above.  Persons  on  this  roster  at  end  of 
tenth  week  will  graduate  with  their  class  and  be  reassigned  as  a group. 
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Sample  Answer  (3)  (cor.'t) 


2.  Instructors-- 

a.  Group  instruction  - Classes  presented  to  groups  will  be  structured 
in  accordance  with  the  lesson  plan  and  course  syllabus.  Instructor 
will  prepare  the  classroom  as  outlined  in  instructor's  guide.  Tho 
conference  method  of  instruction  is  the  preferred  mode  for  group 
instruction  and  the  instructor  may  ask  for  student  participation  in 
subject  areas  listed  as  a study  assignment  on  the  class  schedule. 

b.  Self-paced  instruction  - Instructors  designated  to  monitor  self- 
paced  classes  will  be  available  at  all  times  during  the  scheduled 
period  to  monitor  student  performance,  administer  tests  as  appropriate, 
review  work,  provide  informal  instruction,  and  assist  students  as 
required. 

c.  Study  assignments  - Instructor  may  assign  a reasonablv-j  amount  of  addi- 
tional readings  to  students  experiencing  difficulties  or  to  students 
wishing  to  broaden  their  knowledge  in  a particular  area.  Assignment 
of  additional  readings  in  excess  of  that  prescribed  in  the  course 
syllabus  and  training  scheoule  should  be  considered  in  light  of  the 
student  workload,  nature  of  the  problem,  and  student  preparation  time. 

d.  Additional  instruction  - Instructors  receiving  a request  by  at  least 
five  students  should  report  such  reouests  to  his  acadaric  department. 
The  first  open  hour  will  be  devoted  to  such  instructior,.  Instructors 
are  encouraged  to  honor  requests  by  individuals  if  their  workloads 
permit.  Academic  departments  will  grant  compensatory  tire  for  in- 
structors providing  additional  instruction  outside  the  normal  work  day. 

e.  Administration  of  tests  - Instructors  will  administer  within-course 
tests  as  defined  in  the  syllabus.  Entry  tests,  end-of-phase/end-of- 
course  tests,  and  comprehensive  tests  will  be  administered  by  tne 
Evaluation  Division  with  the  assistance  of  the  ins.ructor.  Security 
of  tests  will  be  as  outlined  in  Instructor's  Guide  and  Evaluation 

Di vision  "OP. 

P.  Class  schedule  - Published  weekly  in  format  of  sample  at  Inci  1. 

Basis  of  issue;  1 each  student;  1 each  instructor;  5,  Director, 

CRTH;  5,  student  company;  15,  academic  depr.rtment. 

3.  information  and  Reports-- 

d.  .'nstructor  summary 

b.  Student  biographical  sum-iary  (profile) 

c.  Counseling  Checxlist  (Seu  Annex  K-Student  Company  SOP) 

d.  Report  of  unsatisfactory  progress 

e.  Summary  report  of  end-of-p lase  tests 

f.  Summary  report  of  end-of-class  acadwnic  standing. 


Sample  Answer  (3)  (con't) 


4.  Support  Personnel -- 

a.  Secretarial  - Requests  for  secretarial  help  will  be  submitted  to  the 
appropriate  academic  department.  Requests  in  excess  of  capabilities 
will  be  forwarded  by  the  academic  department  to  Director,  Resident 
Training  Management. 

b.  Drivers  - Vehicles  and  drivers  will  be  provided  by  Post  Transportation 
in  accordance  vnth  the  training  schedule.  Instances  of  unsatisfactory 
support  will  be  reported  to  the  academic  department  for  appropriate 
action. 

c.  Other  - Other  requirements  will  be  processed  by  academic  departments, 
in.  Facilities  and  Equipment 

a.  Training  aids  - These  will  be  issued  to  the  instructor  by  TASO  not 
earlier  than  48  hours  prior  to  class. 

1)  Training  films  - none  required. 

2)  35mm  slides  - provided  by  instructor,  Kodak  Carousel  projector 
issued  by  TASO  as  above. 

3)  Simulators  - none  required 

b.  Student  equipment  (bring  to  class!  - Issued  at  beginning  of  phase. 

1)  TM  1-225  - Navigation  for  Army  Aviation 

2)  CPU-26  computer 
3!  WEEMS  Plotter 

4)  Packet  of  FLIP  publications  (General  Planning  Manual  and  Low 
Altitude  Enroute  charts) 

c.  Classroom  facilities  - 

l!  Grcjp  instruction  - Room  103,  Classroom  Building  5507 

2)  Self-paced  study  and  individual  work  - Room  104,  Classroom 
Building  5507  (individual  carrels) 

3)  Other  - Requests  will  be  forwarded  to  Director,  RTM  NLT  three 
working  days  prior  to  date  needed 

4)  Maintenance  of  classrooms  - Smoking  and  drinks  not  allowed  in 
classrooms.  Cleaning  will  be  by  contractor. 

IV.  Consumables  and  Courseware 

Basis  of  Issue 


1/ student  (to  be  turned  in  at 
end-of -phase) 
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(3)  (con‘t) 

2.  Study  Guides  and  Workbooks 

. . PT-A-lOU  CPU-26  computer 

b.  Handbook  on  Flight  Planning  Techniques 

PT-A-101-3,  Interpreting  Aeronautical 
Symbol s 


Basis  of  Issue 
1/student 
1/student 
1/student 


0,  Workbook,  Practical  Exercises  in  Problem  l/student 
Solving 


3.  ir;,tructor  Guides 


20/course 


4.  Student  responses  may  take  an  outline  form,  but  must  include  a meaning- 
ful entry  for  each  of  the  9 factors  (A-I).  Responses  should  address 
each  of  the  instructional  settings  selected  in  Block  1.5  and  include 
setting-specific  problems  for  each.  The  Syscem  Master  plan  should  reflect 
aM  necessary  considerations  for  the  implementation  of  the  course.  The 
Enabler  should  judge  the  System  Master  Plan  without  the  use  of  other 
student  products  (so  as  to  determine  if  the  SMP  is  thorough  enough). 

Stud-  who  are  unable  to  generate  a meaningful  response  for  any  factor 
should  reread  pages  138-142  and  try  agair. 


5.  a.  Meet  the  conditions  necessary  for  learning 

b.  Costs 

c.  Resource  availability 


6.  High:  Film,  slide/tape 

Low:  Group  instruction  w/live  instructor,  microfiche,  print 
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The  trainee  should  "accept  partially*'  for  several  reasons. 

First,  it  was  ISD'd,  and  second,  the  instruction  for  the  first 

learning  objective  was  very  effective.  The  instruction  for 

the  second  one  may  need  revisions. 

The  "accept  partially"  decision  is  based  on: 

a.  the  lower  effectiveness  on  the  second  learning  objective 

b.  the  fact  that  only  "Lesson  3"  is  pertinent  to  the  learning 
objectives  and, 

c.  the  materials  are  print  rather  than  slide- tape.  The  last 
makes  it  possible  to  easily  use  only  the  required  subset 
and  to  make  revisions  in  the  second  objective.. 


The  answer  here  is  dependent  on  '-.he  situation.  It  should  in- 
clude, where  aoplicable,  specific  similar  courses  in  other  services, 
government  agencies,  industry,  allied,  military,  other  cofmercial 
sources,  and  schools  or  universities. 


3.  The  possible  variations  in  responses  are  almost  infinite.  The 
match  between  available  materials  and  the  student's  selected 
objectives  will  be  the  determining  factor.  Some  reasons  for 
acceptance  or  rejection  are  listed  below. 


objectives 


intended 

users 


Rejection 

mismatch 
not  available 

underqualified 

overqualified 


Acceptance 


good  match 


similar  or  same 


criteria  or  standards  too  nigh 
standards  standards  too  low 
data  unavailable 

tests  mismatch,  unavailable 


similar  or  same  criteria 
or  standards 


similar  or  same,  could  be 
modified 


delivery  unavailable,  cannot 

system  be  modified 

ownership  cannot  be  changed 

permission  to  use  cannot 
be  obtained 


available  or  obtainable 
modifiable 

can  be  modified,  no  copy- 
•eights  or  fees 


J- 
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Rejection 

level  of  too  much,  too  little 
detail 

cost  of  too  high  - e.g.,  higher  than 
reproduc-  alternatives 


Acceptance 


acceptable  or  similar  to 
needs 


management  unabailable,  unworkable  for 
plan  requirements 

time  too  long 


useable,  alterable 


within  range 


4.  a.  The  Enabler  should  designate  a portion  of  the  learning  obj’ectives 
for  this  part  of  the  exercise.-  Approximately  6 closely  related 
objectives  should  be  sufficient.  The  student's  product  should  be 
judged  on  its  appropriateness  rather  than  its  artistic  value.  The 
script,  sketches,  etc.,  need  only  be  sufficiently  clear  in  order 
for  the  Enabler  to  know  what  the  student  is  trying  to  di. 

(See  Sample  Answer  pages  3-10.) 

b.  For  this  exercise,  the  Enabler  should  designate  4-10  objectives  for 
which  existing  materials  are  available.  The  student's  product  should 
follow  the  guidelines  given  on  page  250  of  the  Phase  III  -Manual. 


The  instructor's  guide  outline  should  include  the  items  described  on 
pages  262-2fi5  of  the  Phase  III  Manual. 


6.  Jo  the  materials  match  your  learning  objectives  and  standards,  your 
intended  students,  will  it  cost  less  to  use  or  be  available  sooner  than 
what  you  could  develop? 
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Sample  Answer  (4a) 

FRAME  1 - RULES  FOR  DETERMINING  ETA 


RULE 


MAP  A 


CAAF 


Leg  1 + 
+ Leg  4 


1.  Measure  distance  of  each 
leg  of  flight.  Distance 
given  in  naatical  miles. 


2.  Calculate  Estimated  Time 
Enroute  (ETE)  for  first 
leg  of  flight  by  dividing 
distance  (from  Rule  1) 
by  ground  speed.  State 
ETE  in  hours  and  minutes. 

Repeat  Rule  2 for  each  le 


3.  Sum  the  ETE's  for  all  legs 
of  the  flight 


To  the  sum  found  in  Rule  3,1 
add  15  minutes  for  aircraft 
warmup  and  takeoff. 


5.  Add  the  result  found  in 
Rule  4 to  the  pilot's 
takeoff  time.  Pilot's 
takeoff  time  is  stated 
in  ZULU  (not  local)  time. 
The  result  is  the  pilot's 
ETA  (Estimated  Time  of 
Arrival). 


6.  Compare  your  ETA  to  pilot's. 
If  times  do  not  agree  within 
± 3 minutes,  recompute. 


HAP  LEGEND:  - Airway  Intersection 


15  = ^ hour 
65"  (15  minutes) 


ALPHA 


Leg  2 + Leg  3 
' Total  ETE 


Total  ETE  + 15  minutes 


+ Takeoff  Time 


ZULU  time) 


(ETA  in 


Takeoff  Time; 
1400  ZULU 
(1400Z) 

Ground  Speed: 
60  knots 
(60K) 


?.) 


Leg  3) 


CHARLIE 


(Leg  4) 


@ HGM  (lAF) 
VOR 


T 

0) 

3 

/> 


MGM 

Oannelly  Field 
(destination) 


V-7,etc.  - Victor  Airway 
lAF  - Initial  Approach  Facility 
VOR  - Visual  OMNI  Range 


The  first  step  in  calculating  ETA  is  to  dete’Tnine  the  distance  for  each 
leg  of  flight  and  between  the  last  intersection  crossed  and  the  initial 
approach  facility  serving  as  an  approach  facility  for* the  point  of  intended 
landing.  You  must  carefully  apply  rules  from  FRAME  1,  since  a discrepancy 
of  more  than  3 minutes  may  allow  the  pilot  to  takeoff  with  an  insufficient 
fuel  reserve.  Refer  to  map  A. 


The  total  distance  for  Leg  1 and  Leg  2 is 


nautical  miles  (NM), 
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Sample  Answer  (4a)  (con't) 


FRAME  3 


Answer:  45  nautical  miles 


FRAME  4 


Rule  2 provided  you  with  a means  of  computing  the  Estin^ted  Time  Enroute  (ETE) 
for  a leg  of  flight.  Applying  this  rule  to  detenriine  the  ETE  for  the  first 
leg  of  the  flight  from  CAAF  to  MGM  (Map  A)  would  result  in  this  equation. 


' - ^Mhours) 

Based  on  a ground  speed  of  60  knots  your  ETE  for  Leg  1 would  be 


TURN  TO  FRAME  5 (NEXT  PAGE) 
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Sample  Answer  (4a)  (con't) 


FRAME  5 


Your  Answer;  h hour  or  15  minutes. 


Remember: 


Distance  = 15  - 
Ground  Speed  ^ 


h hour 


FRAME  6 


To  determine  the  Estimated  Time  Enroute  (ETE)  for  the  flight  from  CAAF  to 
the  Initial  Approach  Facility  (Rule  3)  you  would  repeat  the  procedure  used 
in  FRAME  3 for  each  leg  of  flight.  NOTE:  It  is  important  to  compute  these 

ETEs  individually  since  a change  in  heading  may  change  head  winds  and  change 
your  ground  speed.  Applying  this  rule  would  result  in  a total  ETE  of 

for  the  flight  from  CAAF  to  M6M  VOR  (lAF). 
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Sample  Answer  (4a)  (con't) 


FRAME  7 


Your  answer:  1 hour  and  45  minutes 

Computations;  1st  Leg  - ^ = k hour 

60 

2nd  Leg  - 30  = h hour 

3rd  Leg  - 45  = 3/4  hour 
60 

4th  (final)  Leg  - = k hour 

60 

Total  ETE  - 1 3/4  hours  or  1 hour  and  45  minutes 


FRAME  8 


You  are  not  yet  ready  to  compare  your  ETA  with  the  pilot's  ETA.  Remember 
Rule  4 - YOU  MUST  ADD  15  MINUTES  TO  THE  TOTAL  ETE  for  engine  warmups  and 
takeoff.  THEN  ADD  THIS  TOTAL  TO  PILOT'S  TAKEOFF  TIME. 

YOUR  ETA  '^OR  THE  FLIGHT  DEPICTED  ON  MAP  A IS . 

A.  ETA  2 Hours  (Turn  to  FRAME  11) 

S.  ETA  1600  Zulu  (1600Z)  (Turn  to  FRAME  17) 
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Sample  Answer  (4a)  (con't) 


FRAME  9 


What  you  failed  to  do  was  to  apply  Rule  5 which  states,  "Add  the  total  ETE 
derived  from  application  of  Rules  3 and  4 to  the  departure  (take  off)  time 
for  the  flight.  The  result  is  ETA  expressed  in  ZULU  (Greenwich  Mean  Time). 
Go  back  to  FRAME  c and  select  the  correct  answer. 
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ENABLER  FEEDBACK 
MODULE  10 


1.  The  student's  analysis  should  include: 

Pretest:  All  students  were  naive. 

Entry  Test:  All  students  had  the  necessary  entry  skills. 
Within-Course  and  Posttest: 


I ten. 

(1)  The  third  within-course  test  may  have  been  unnecessary. 

Try  deleting  the  last  leari.ing-testing  sequence. 

(2)  The  second  within-course  test  may  have  been  unnecessary. 

Try  deleting  one  of  the  learning-testing  sequences. 

(3)  Students  did  not  master  the  material  but  passed  the  posttest. 
No  change  recommended. 

(4)  Students  did  not  do  well  on  the  third  trial.  Revise. 

(5)  Students  did  well  on  first  attempt  within-course  and  failed 
posttest.  Maybe  a review  before  the  end  would  help. 

(6)  Again  too  much  loss  before  the  posttest.  Also  recoimiend  a 
review. 

(7)  Improvement  shown  within  course  but  lost  before  postcest. 
Needs  more  or  improved  instruction. 

(8)  Students  failed  within-course  and  posttests.  Revise. 

(9)  The  students  did  worse  on  the  third  try  than  on  the  first 
two.  Some  confusing  or  misleading  instruction  should  be 
suspected  here. 

(10)  rnHure  both  times.  Revise. 

The  student's  rationale  can  very,  but  inconsistencies  such  as  in  (9) 
should  be  noted. 


2.  a.  Individual  trials  are  discussed  in  Block  III. 5 on  pages  285-303. 

The  trainee  should  have  as  a minimum:  Answer  sheets  for  entry  test, 
the  pretest,  the  posttest,  and  the  wi tr, in-course  test  items,  list  of 
comments  and  questions  from  the  student  and  time  required  for  indi- 
vidual trials.  No  specific  format  is  required  but  it  should  be 
orderly  enough  to  be  useful  to  other  people. 
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b.  The  test  item  data  In  "a"  should  be  coded  by  objectives  and  tied  to 
the  specific  pages  of  learring  materials.  The  student  should  list 
the  deletions,  replacer’ents,  and  additions 

(See  Sample  Answer  on  pages  3 and  4.) 

c.  The  revisions  and  rationale  for  them  shc:ld  be  documented. 

d.  The  results  of  the  second  trial  should  be  summarized  in  the  same 
way  as  in  "a.'‘ 


Sample  Answer 

DATA  FROM  2nd  TRIAL;  Postt^ 

Item  Student  B 

1 - 2 answers 

2 + 

3 + 

4 + 

b + 


+ « Correct 
- ■ Incorrect 


6 + 

Suirenary  of  Analysis;  Delete  or  change  item  1 on  test.  There  are  2 

correct  answers. 


■k-kit 


H is  an  'terative  process  of  trying  instruction  on  students  fr'^m  the 
intended  DO;..lation  using  their  responses  to  revise  it,  then  trying  it 
out  on  i;r.'r9  students,  until  they  reach  some  preset  cr i ».f.rion  or 
intended  outcome. 
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Sample  Answer  (2b) 

DATA  FROH  1st  TRIAL: 

Posttast  + * Correct 

- * Incorrect 

Item  Student  A 

1 

2 + 

3 

4 + 

5 + 

6 

Summary  of  Analysis:  Clarify  Rule  5.  Revise  Frame  7.  Add  another 

Frame  on  calculating  ETA. 


{SF.E  TEST  ITEMS  NEXT  PAGE) 
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Sample  Answer  (2b)  (con't) 

T “T  ITEMS 


1.  Estimated  time  of  arrival  of  3:00  o'clock  P.M.  would  be  expressed 

A.  1500 

B.  1500  hours 

C.  3:00  o'clock  P.M. 

D.  1500Z 

2.  Estimated  time  enrouta  for  a leg  of  flight  can  be  determined  by 
which  of  the  following  equations; 


^ Distance 

r\ « 

True  airspeed 

g Distance 
Grounds peed 


3.  To  allow  sufficient  time  for  aircraft  warmup  and  take  off,  it  is  necessary  to 

A.  add  15  minutes  to  total  FTE 

B.  add  30  minutes  to  total  ETE 

C.  subtract  15  minutes  from  total  ETE 

D.  none  of  the  above 

^ if,  after  having  calculated  the  ETA,  you  find  that  your  ETA  is  10  minutes 
greater  than  the  pi‘'ot's  ETA  yOu  would; 

A.  Diaspprove  the  flight  plan 

B.  Insist  that  you  are  right 

C.  Both  you  and  the  pilot  ••.ould  recompute 
0.  Both  A and  B 

b.  Estimated  time  enroute  (ETE)  is  expressed  in; 

A.  Miles  per  hour 

B.  Zulu  time 

C.  Hours  and  minutes 

0.  Unnecessary  since  you  <:re  ccnccrncd  with  ETA 

6.  Witn  a total  estimated  t’me  enroute  (F.~F)  of  2 hours  and  a departure  time 
of  1500Z,  the  ETA  shoulo  be; 

A.  1300Z 

B.  1515Z 

C.  1715Z 

D.  None  of  the  above 


=»  ETE 
= ETE 


kic 
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ENABLER  FEEDBACK 
MODULE  11 


1.  The  student's  response  should  identify  or  describe  such  items  as 
the  following: 

a.  Instructor's  manual 

b.  Student's  inanual 

c.  Instructor  training 

d.  Tests 

e.  Evaluation  plan 

f.  Instructional  materials 

g.  Time  allocation 

h.  Equipment 

1.  Space  allocation 

NOTE:  The  student  may  say  that  he  cannot  complete  this  exercise  until 
he  Joes  the  exercises  in  Block  V.l  (Module  12).  If  so,  encourage  him 
to  complete  Block  V.l  and  then  come  back  to  this  exercise. 

Student's  responses  should  be  sufficiently  complete  and  detailed  that 
the  selected  "instructor,"  given  these  items,  likely  could  effectively 
fill  the  instructor's  role.  The  items  above  are  discussed  in  Block 
IV. 1,  pages  4-15. 


Sample  Answer 

The  instructor  should  have  the  following: 

a.  Instructor's  manual 

b.  Details  of  internal  evaluation  plan  that  affect  the  student  (instructor). 

c.  Details  of  the  management  plan  that  affect  the  instructor  - local  SOP, 

scheduling,  equipment  location,  contingency  plan-  and  other  similar 
details 

d.  Instructor  training: 

1)  Movement  and  gesturing  techniques 

2)  Oral  techniques  and  manner  of  delivery 

3)  Classroom  preparation 

4)  Use  of  equipment  and  media 

5)  how  to  develop  an  effective  training  situation 

6)  Handling  student  questions 

7)  Use  of  training  aids 

8)  How  to  improve  instruction 


'k'k'k 
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The  emphasis  here  should  be  on  local  requirements  not  covered  in  the 
student's  instructions  prepared  in  Block  III. 4.  These  instructions 
should  be  tailored  to  the  particular  instructional  setting,  activities, 
etc.,  included  in  the  particular  instruction  developed  in  this  workshbp 
in  Block  III. 4.  Some  or  all  of  the  following  may  be  appropriate: 

a.  Tell  the  class  where  they  will  meet,  when  they  will  meet,  where  the 
bus  will  pick  them  up,  how  much  time  they  will  have  for  lunch,  where 
they  may  eat,  and  so  on. 

b.  Describe  the  type  of  preparation  the  students  must  make  before 
coming  to  class.  Will  there  be  nomework?  How  often  and  how  much? 

Will  they  have  to  bring  tool  kits,  or  bathing  suits,  or  wear  fatigues? 

c.  Explain  classroom  procedures.  Will  the  course  be  self-paced  or 
group  instruction?  Who  are  the  instructors?  How  long  will  the  course 
last?  What  are  the  grading  procedures?  What  will  tests  cover  and  how 
will  they  be  administered? 

d.  Since  this  course  is  still  in  the  development  stage,  student  coopera- 
tion is  especially  important.  The  studencs  must  be  patient  when  as 
yet  untried  instruction  fails  to  perform,  or  when  student  evaluations 
are  requested.  Explain  to  the  class  that  it  is  a partner  in  tne 
development  of  the  course  and  all  cooperation  and  Involvement  will  be 
appreciated. 

Student  responses  should  include  the  types  of  activities  listed  above, 
if  a student  is  unable  to  produce  such  a list  on  his  own,  show  him  yours. 
Then  oermit  him  to  re-submit  a list. 


The  platform  instructor  must  plan  and  deliver  most  instruction,  while 
the  self-paced  instructor  usually  has  his  instruction  planned  and 
delivered  for  him. 

The  platform  instructor  must  be  skilled  ac  holding  and  communicating 
with  the  slower  student. 


c.  The  self-paced  instructor  must  assure  that  students  move  through  the 
instruction  at  a reasonable  pace. 

d.  The  self-paced  instructor  must  decide  who  must  be  "recycled"  through 
what  ins true  cion  and  when. 

e.  The  platform  instructor  must  base  his  decisions  on  the  good  of  "most" 
of  his  class.  The  self-paced  instructor  must  decide  what  is  best  for 
each  individual  in  his  "class." 

f.  The  platform  instructor  usually  conside»'s  only  one  topic  at  a time. 
The  self-paced  instructor  must  be  prepared  to  entertain  questions 
about  any  pertinent  topic  at  any  time. 


g.  The  Instructor  in  the  self-paced  course  must  be  thoroughly  familiar 
I with  the  planned  management  of  the  instruction.  The  Dlatfcrm  instruc- 

\ tor  must  be  thoroughly  competent  in  the  subject-matter  area  Le'ng 

taught. 


Student  lists  should  contain  the  types  of  teacher  behaviors  listed  above. 

Students  who  are  unable  to  develop  such  lists  should  reread  Bloc!:  IV.  1 

and  try  again.  | 

n 

Sample  Answer  ^ 

Self-paced  Course: 

Instructor  acts  as  teacher,  monitor,  tutor  and  fulfills  an  administrative 
need  by  keeping  track  of  student  progress,  recommending  or  providing 
remedial  instruction.  He  will  occasionally  have  to  assist  students  with 
personal  problems  by  providing  guidance  and  counselling.  He  is  a manager, 
tutor,  counselor,  supervisor,  and  actainistrator.  His  job  is  much  more 
complex  and  demanding  than  the  average  platform  instructor's. 

Platform  Instructor: 

Primary  function  is  to  present  instruction.  He  probably  interacts  less 
with  students  and  is  primari‘*y  a lecturer. 

Problem  Areas  for  Instructors: 

1) *  Identify  slow  learners  —Provide  remedial  instruction  or  tutoring. 

2) *  Identify  fast  learners  —Allow  them  to  progress  (if  possible)  to 

maintain  interest  level  and  prevent  boredom.  } 

3)  Deficiencies  or  problem  areas  with  instructional  materials--  Record  1 

and  provide  corrective  action  if  possible. 

4)  Unexpected  situations— Be  prepared,  have  contingency  plans  available. 

5)  Adequacy  of  materials— Insure  they  are  available,  check  and  re-check. 

6)  Equipment  malfunction  or  breakdown— Plan  ahead.  Know  where  and  whom 
to  call. 

*Applicable  more  to  self-paced  than  platform  instructors. 

icifir 


4.  If  an  Instructor  does  not  document  problems,  they  are  . ,•  -sly  to  come 

to  the  attention  of  those  with  the  ability  to  solve  th  m. 

If  an  instructor  makes  a change  in  1 s course  without  doa  ..Minting  it, 

others  have  no  way  to  evaluate  the  isults.  Was  suuse'.,ue-!-;  success  or 
failure  due  to  the  change?  Or  was  it  in  spite  of  the  cH-.nge?  There  is 
no  way  to  tell  without  documentatv.-r.. 


Students  who  are  unable  to  describe  these  reasons  should  T-ead  pages 
35-37  of  the  Phase  IV  Manual  and  try  again. 


5.  The  course  designer,  manager,  and  instructor  use  the  instructional 
management  plan  to  implement  the  instructional  program. 


6.  Hard  data  is  objective;  e.g.,  test  results,  time,  etc. 
Soft  data  is  subjective;  e.g.,  opinions,  attitudes,  etc. 
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ENABLER  FEEDBACK 
MODULE  12 


1.  An  items  1r.  the  plan  should  be  complete,  clea.-  and  logical.  The 
Internal  evaluation  plan  Is  discussed  on  pages  11-33  of  the  Phase  V 
Manual.  The  individual  items  are  discussed  on  the  following  pages: 

a.  Progress  evaluation  plan:  pages  11-14 

b.  Process  evaluation  plan:  pages  14-22 

c.  Performance  evaluation  plan:  pages  22-28 

d.  A plan  for  collecting  information  from  students:  pages  29-30 

e.  A plan  for  collecting  information  from  instructors:  pages  >.1-33 

If  any  part  of  the  plan  is  inadequate,  tell  the  student  which  areas 
are  inadequate  and  have  him  read  the  appropriate  sections  in  the  manual 
and  revise  that  portion  of  the  plan. 


{See  Sample  Answer,  pages  2-6.) 


2.  ■^he  emphasis  in  this  exercise  should  be  on  correct  interpretation  of 
the  data  and  appropriate  recomnendations  for  revision.  Some  guidelines 
are: 

a.  For  Unit  1,  the  entry  test  scores,  pretest  scores,  posttest  results, 
time  requirements,  and  student  opinions  appear  realistic  and  In 
accordance  with  the  plan.  No  revisions  are  required. 

b.  The  average  entry  test  score  of  32  for  Unit  2 is  unusually  low  for 
a course  with  no  renedfal  lessons.  Table  c indicates  that  students 
with  low  Unit  2 entry  test  scores  did  poorly  on  the  Unit  2 instruc- 
tion. There  is  a strong  indication  here  that  the  student  entry 
behavior  is  not  according  to  plan.  This  is  supported  by  the  instruc- 
tor's report  of  having  to  provide  additional  instruction,  by  the 
unexpectedly  high  time  requirement,  by  the  low  mmfcer  passing  the 
lesson  and  unit  posttests,  and  by  student's  opinion  of  the  instruc- 
tion. The  Project  Schedule  shows  that  entry  behavior  was  checked 
using  an  inappropriate  substitute  group.  The  summary  test  data 

and  student  performance  data  show  that  they  entry  level  was  set  too 
high  for  the  students. 

The  reconinendation  for  revision  should  be  that  the  learning  analysis 
for  the  Unit  2 learning  objectives  be  extended,  that  test  items  be 
developed  for  the  new  learning  cAjectives,  and  that  the  new  assumed 
entry  behavior  be  verified  on  an  appropriate  group  of  students.  New 
instruction  will  have  to  be  Iccated  or  developed  for  the  new  learnioo 
objectives,  and  the  validation  step  and  internal  evaluation  step  will 
have  to  be  repeated. 


Sample  Answer  (1) 

Internal  Evaluation  Plan  for  US  Army  Aviation  Center 
Proponent  MOS's  Process  and  Progress  Evaluation  Plan 

I . Assumptions: 

1.  An  jobs  are  being  performed  - no  new  jobs  to  be  concerned  with. 

2.  There  are  existing  training  programs  for  each  job. 

A . '-Vogress  Evaluation 

I . Information  Plan  : 

1.  Information  Requirement  and  Reports  (see  Inclosure  1). 

2.  Project  Time  Schedule  (see  Inclosure  2). 

3.  Collection  of  information  (see  Instructional  Management  Plan  - Incl.  3). 

I I . Responsibilities  : 

1.  See  Inclosure  1 (Information  Requirements  end  Reports) 

2.  USAAVNC  Regulation  10-1  — Operations  and  Functions  Manual 

B.  Process  Evaluation 

I . Responsibilities  ; 

1.  USAAVNC  Regulation  10-1 

2.  Information  Requirements  and  Reports  (Incl.  1) 

II.  Procedures  - Inclosure  4 (Process  Evaluation  Checklist) 

(Tee  Table  V.2  Phase  IV  and  V Manual) 

Performance  Evaluation  Plan  (Inclosure  5) 


t of  Major  Milestones  Completed* 


Information  Requirements  & Repo»"ts 
(Incl.  1) 


(Effect.ve  December  1,  1975) 
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i* . 


(Incl.  4) 


BLOCK  I.  1 Analyze  Job  Process  Evaluation  Checklist 

Completed  Procedure  Explanation 

Review  Available  Job  Data  

Plan  Data  Collection 

Train  Job  Analysts  

Prepare  Forms  

Select  Sample  

Collect  Data  at  Job  Site  

Analyze  Data  

Revise  Consolidated  List 


BLOCK  I.  2 


I 

I 

» 

i 


BLOCK  II.  1 


I 


no 


(Incl.  5) 

Performance  Evaluation  Plan 


External  Requirements 

1.  Documentation 

a.  Trip  Reports  (On-Site  Interviews  with  CD's) 

b.  CODAP 

c.  Reports  of  ARTEP  Evaluations 

d.  SQT  Results 

2.  Summary  of  Data 
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Entry  Skills 

1.  Student  Profile 

2.  Summary  of  Entry  Test  Results 

3.  Battery  of  Aptitude  Tests  - Applicable  Area 


Performance  on  Internal  Tests 

1.  Weekly  Suimiary  of  Student  Performance 

2.  Counseling  Checklist  {see  Student  Company  SOP) 

3.  Report  of  Unsatisfacco^’y  Student  Progress 

4.  Summary  Report  of  Class  Acaoe-'ic  Standing 


IV.  Instructional  Units  - see  Appropriate  Syllabus  or  Ijp 


The  Project  Schedule  and  the  results  of  the  original  course  valida- 
tion also  should  be  reviewed  to  find  out  why  the  problem  with  Unit 
2 was  not  detected  earlier.  This  is  to  make  certain  that  any 
mistakes  made  the  first  time  wil"'  not  be  repeated. 

c.  The  average  pretest  score  of  41  on  Unit  3 indicates  that  most 
students  already  know  the  material.  This  conclusion  is  reinforced 
by  the  large  number  pa'-sirg  the  lesson  and  unit  posttests,  and  by 
the  unexpectedly  short  average  time  to  complete. 

A recommendation  should  be  made  that  Unit  3 be  revised.  Either 
the  learning  objectives  that  most  students  already  know  should  be 
deleted  from  the  course,  or  the  pretest  should  be  used  to  exclude 
the  students  from  the  parts  they  already  know. 

(See  Sample  Answer,  page  8.) 


Student  responses  must  contain  most  of  the  questions/categories  listed 
under  "determine  who  wi’l  provide  data,"  pages  65-66i  "detenriine  what 
data  are  required,"  pages  66-68;  "determine  when  external  evaluation 
will  take  place,"  page  68;  and  "determine  how  the  data  will  be  gathered," 
pages  68-69.  Incomplete  student  responses  must  be  completed  and  re- 
submitted. 

(See  Sample  Answer,  page  9.) 


The  emphasis  in  this  exercise  should  be  on  correct  interpretation  of 
the  data  and  appropriate  recommendations  for  revision.  Some  guidelines 
are: 

a.  Although  students  felt  only  moderately  confident  in  their  prepara- 
tion for  tasks  1-30,  Unit  1,  all  other  sources  indicate  that  trainees 
were,  in  fact,  strong  in  those  tasks.  Thus,  as  in  the  internal 
evaluation  in  Block  V.l,  no  revision  of  Unit  i is  recommended. 

b All  data  sources  indicate  that  Unit  2 tasks  are  a problem  area.  The 
svsluation  team  report  suggests  that  the  "old  training  system  may 
have  been  more  effective  in  teaching  these  tasks.  These  results 
support  the  revision  recommendations  made  in  Block  V.l,  Extend  the 
learning  analysis  of  the  Unit  ? objectives,  add  objectives  and  addi- 
tional instruction. 

In  addition,  the  original  task  list  seems  to  be  out  of  date.  It- 
includes  tasks  that  are  no  longer  performed  because  of  the  compure.-- 
i?ed  inventory  system.  It  also  omits  che  new  tasks  associated  with 
the  computerized  system.  Revisions  must  include  eliminating  training 
for  unnecessary  tasks  and  adding  training  for  new  tasks. 
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Sample  Answer  (2) 


IN.ER 


Interpretation  of  Test  Data 

1.  Entry  Skills 

Unit  1 - (45)  acceptable  level  of  pei^formence 
Unit  2 - (32)  unacceptable  level  of  perforaance 
Unit  3 - (49)  acceptable  level  of  performance 

2.  Pre-tests 

Units  1 and  2 - low 
Unit  3 - high 

3.  Post  Test 


Unit  1 - 2 of  100  no-go 

Unit  2 - 10  of  100  no-go  (After  3 trials) 

Unit  3 - 100%  go 


Findings 

1,  Students  in  general  do  not  have  requisite  entry  skill  for  Unit  2 
7 Unacceptable  failure  note  on  Unit  2 

3.  Ave"age  time  spent  on  Unit  2 is  unacceptable 

4.  Attitude  of  students  is  poor  to  fair  in  Unit  2 


Recommendations  for  Revision 


1.  Revise  Unit  2 - Based  on  data  in  Table  a - Unit  2 may  not  be  relevant 

to  group  wtio  went  through  - Take  a close  look  at  unit  - Complete 

Front  End  Analysis 

2.  Revise  Policy 

a.  Allow  advanced  placement  for  high  pre-test  scorers  or  exempt  where 
applicable 

b.  Provide  remedial  work  ss  required 

c.  Look  at  objectives  for  Unit  2 - May  not  be  consistent  with  the 
job- standards.  May  be  too  high  - May  require  different  delivery 
system 

d.  Lock  at  sequence  - May  want  to  p’ace  Unit  3 first,  then  Unit  1, 
then  Unit  2 if  there  is  a potential  for  transfer  of  learning 

G.  Use  control  groups  to  identify  problem  area  - Select  by  random  sample 
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Sample  Answer  (3; 

External  Evaluation  P~’cn 

A.  Data  Sources 

1.  Baseline  data  from  tryout  of  TLOs 

2.  Commanders  and  supervisors 

3.  Evaluation  team  (field  trips) 

4.  Job  incumbent  of  graduates 

5.  Graduates  of  an  existing  course 

B.  What  Data  Are  Required 

1.  On-the-job  performance  after  graduation 

2.  Improvement  in  on-the-job  performance  after  new  course 
(comparison  of  previous  graduates  with  new  graduates) 

3.  Change  ‘-n  OJT  training  requirements  (old  vs  new) 

4.  Recommendations  for  change  in  instruction 

I 

i 
t 

C.  r ,er.  External  Evaluation  Will  Take  Place 

Within  30-90  days  after  graduation  - Conti  =uous  process  for  sample  of 
students  from  each  class 

0.  Data  Collection  Methods 

1.  n.-i-bits  administration  of  JPM 

2.  COOAP/MODB 

3.  Questionnaires 

4.  On-site  or  telephonic  in-Le*’*- -ews  of  cojmi=>rders  and  supervisors 
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(con't) 


All  data  sources  indicate  that  the  instruction  in  Unit  3 is  too 
easy;  everyone  seems  to  know  most  of  the  tasks  prior  to  instruction. 
The  entry  level  line  established  for  the  learnino  objectives  of  many 
of  those  tasks  must  be  raised.  This  will  eliminate  training  on  many 
objectives. 


S iple  Answer 


Interpretation  of  Data: 


1.  Baseline  data  for  Unit  2 indicates  satisfactory  performance  on  old 
instructional  system. 

2.  Graduates  in  general  reflected  dissatisfaction  with  level  of  training 
in  Unit  2. 

3.  Based  on  student  comments  and  iclitudes  reflected  in  INER,  Unit  2 
does  not  reflect  real-vaorid  requirements. 

4.  Supervisor  comms'^ts  bear  out  #3. 

Recopgriendaticns: 

Appa»-ent  discrepancy  identified  between  training  needj  and  actual  training 

provided.  Analyre  job  with  respect  to  stock  inventory  system  and  revise 

Unit  2 as  necessary. 

Provide  Unit  4 (computer  inventory  system)  for  trainees  successfully 

passing  entry  test  for  Unit  3 and  high  pretest  on  #3. 


Each  item  on  the  revision  plan  format  gi'-en  in  Figure  V.16  of  Block  V.3 
should  be  complete.  The  primary  emphasis  should  be  on  the  appropriate- 
ness of  the  revision  plan  based  on  the  data  and  assumptions. 

(See  Sample  Answer,  pages  11  and  12.) 

Ti.  Job  performance  is  inadequate.  Changes  in  job  requirements.  Changes  in 
entry  behavior  of  students. 

/.  Not  as  much  chance  as  on  the  first  and  second  time.  There  tS  a point  at 
which  further  improvement  in  any  one  factor  (time,  cost,  effectiveness, 
etc.)  gets  more  and  more  difficult  and  expensive. 


8.  No,  the  ISO  process  never  stops.  The  reasons  are  given  on  page  83  of 
Phase  V. 
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Sample  Answer  (5) 

Revision  Plan  for  Flight  Operations  Specialist  Course 

1 .  Training  Rgoui remen ts: 

a.  Total  number:  250  per  year  (iO  classes  of  25  each) 

b.  Degree  of  proficiency:  Adequate  performance  on  JPM  within  90  days 
after  graduation. 

c.  MOS:  71P 


2.  Costs: 


a.  Institutional:  $1250  per  student 

b.  OJT:  350  per  student 

c.  Total:  $1600  per  student 

3.  Reason  for  Revision: 

a.  Based  on  external  evaluation 

b.  Shift  training  burden  from  unit  to  institution  at  an  overall  savings. 

Details  of  Oeficiency:  Graduates  unable  to  perform  at  an  acceptable 
Isvcl  on  JPM.  EXER  report  indicates  that  equipment  ’n  sufficient  quanti- 
ties .5  not  available  to  support  FOOT  program.  Also  lack  of  qualified 
supervisors.  Change  in  job  requirements. 

5 . Deficiency  i n Present  Course: 

a.  CPU-26  V.  -ter  not  taught  in  present  course 

6 Recommended  Changes: 

Develop  unit  on  CPU-26  computer  based  on  reports  from  field  evaluation 
teams  and  CCDAP.  Delete  unit  on  terminal  weather  reports  and  develop 
JPA  and/or  STEP. 

a.  Estimated  Lost  of  Revision:  Inc.'ease  in  instructional  costs  of 

$150  per  student  for  instructional  materials  & instructor  time.  No 
additional  costs  for  student  TDY  since  course  length  will  not  be 
changed.  Potential  unii;  savings  of  $250  per  student  through  elimina- 
tion of  FOOT  program  for  compute**. 

b.  Time  Schedule:  See  internal  evaluation  plan 

c.  Impact:  No  major  Impact  on  course  length.  No  increase  in  instructor 

spaces. 


> t 

•1 

I 


d.  Personnel  Requirements:  No  change 
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Sample  Answer  (5)  (con't) 


7.  Estimated  Costs: 

a.  Institutional:  $1400  per  student  (increase  of  $lfi0) 

b.  Unit:  100  per  student  (decrease  of  $250) 

c.  Total:  $1500  per  student  (estimated  savings  of  $100  per 

student) 

8.  Alternative  Plans  Considered:  FOJT  program  supplemented  by  STEP.  Based 
on  Hack  of  supervisor,  difficulty  of  material,  and  availability  of 
resources,  resident  school  plan  seems  better. 

9.  Future  Requirements:  Outlook  is  stable.  No  anticipated  changes  in  job. 

10.  Probable  Results  if  Course  Not  Revised; 

a.  Less  than  adequate  performance  for  on-the-job  performance  of 
graduates. 

b.  Dissatisfaction  with  school  product  on  part  of  comnanders. 

c.  Poor  unit  performance  on  ARTEP  evaluation. 

0.  Poor  individual  performance  on  5QT. 

11.  Assumptions: 

a.  Baseline  data  has  been  collected. 

b.  Funds  are  available  for  External  Evaluation. 

c.  Current  CODAP  reports  are  available. 

d.  Previous  course  not  developed  using  ISD. 

e.  JPMs  for  the  job  will  be  developed. 
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The  multiple-choice  pre/pcsttests  have  a coded  page  nunber  in  the 
upper  left-hand  corener  which  designates  that  t.iey  are  Technical  Workshop 
materials  and  which  phase  of  the  IPISD  Manuals  they  cover. 


PRE/ POSHEST—  TW 

t 

Technical 

Workshop 

material 


?PI 

t 

Pre/Posttest 
Phase  I 


1 


t 

Page  # 


For  example,  TW-PPIII-3,  is  the  third  page  of  the  pre/posttest  material  for 
Phase  III  of  the  IPISD  Manuals.  . When  these  tests  are  grouped  according  to 
phases  they  serve  as  the  pretests  for  the  workshop. 

These  tests  can  also  be  broken  down  into  tests  which  cover  a particular 
modules.  They  are  then  used  individually  as  posttests  for  module  assignments. 

P-H^se  module 


Phase  I: 


Phase  II: 


Phase  III: 


Phase  IV: 


Module  1 
Module  2 
Module  3 
Module  4 

Module  5 
Module  6 

Module  7 
Module  8 
Module  9 
Module  10 

Module  11 


Phase  V: 


Module  12 
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The  answer  keys  list  the  answer  for  each  question  and,  where  appli- 
cable, the  page  nunfcer  of  the  manual  where  the  answer  may  be  found.  Son« 
questions  will  not  have  a specific  page  reference  as  they  are  general 
questions  covering  a broad  area  of  the  manuals. 

The  copies  of  the  pre/ posttests  contained  in  this  appendix  are  to  be 
used  as  masters  for  duplicating  material  for  the  workshop.  Clost  security 
on  these  tests  and  on  the  answer  keys  should  be  maintained. 
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MODULE  1 


DIRECTIONS:  Using  the  answ  sheet  provided  for  recording  your  responses, 
choose  the  most  appropriate  answer  for  each  item. 


1.  In  order  to  develop  adequate  training  for  a job,  you  must  find  out; 

a.  What  major  duties  the  job  includes: 

b.  What  tasks  make  up  the  job. 

c.  Where  the  job  holdt  s will  be  loacted. 

d.  All  of  the  abcve. 


2.  The  first  step  in  the  ISD  process  it  to: 

a.  Analyze  exi  ^.ing  courses. 

b.  Identify  the  discrepancy  between  present  training  and  present 
or  future  needs. 

c.  Determine  a budget  allocation  system. 

d.  FoHnulate  a set  of  tasks  for  training. 


3.  Which  does  nojt  describe  a job? 

a.  Jet  transport  pilot 

b.  Radar  technician 

c.  Bomb  defuser. 

d.  Military  policeman 


4.  Which  of  the  following  statements  is  false? 

a.  Tasks  are  performed  In  relatively  short  periods  of  time. 

b.  Tasks  are  dependent  on  other  tasks. 

c.  A task  is  observable. 

d.  A task  is  a statement  of  action. 


5.  Which  of  the  following  is  not  documented  in  a job  analysis? 

a . Condi ti ons 

b . Cues 

c.  Standards 

d.  Times 
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6.  Whic"'.  data  collection  niethod  is  likely  to  be  best  in  jinalyzing  a nt 
job?  ~ 


a.  Group  interviews 

b.  Library  search 

c.  Oury-of -experts 

d.  Individual  and  observation  intervi^i^ws 


7.  Questionnaire  surveys  should  sample: 

a.  As  large  a group  as  possibie  c,  at  least  3,000  individuals  in 
a very  large  DOS. 

b.  All  job  holders  at  selected  stations. 

c.  Everyone  who  knows  about  the  job. 

d.  Only  the  mcist  expert  job  incumbenu. 


8.  A validated  task  list: 

•a.  Contains  only  tasks  selected  tor  training. 

b.  May  still  contain  tasks  requiring  no  training. 

c.  Is  useful  only  in  job  analysis. 

d.  Should  be  sent  to  each  individual  interviewed. 


9.  Task  delay  tolerance  refers  to: 

a.  Hov/  long  you  can  put  off  teaching  the  task. 

b.  The  length  of  time  to  do  the  task. 

c.  The  tolerable  delay  between  the  initiating  cue  and  the  task 
performance. 

d.  The  length  of  delay  tolerable  between  teaching  the  successive 
elements  that  make  up  a task. 


10.  Consolidating  data  to  select  tasks  for  tarining  is  for  the  purpose 

a.  comparing  incumbent  and  supervisor  responses. 

b.  Helping  you  to  select  what  tasks  should  be  trained. 

c.  Eliminating  prior  documentation  materials. 

d.  Insuring  that  the  information  can  be  computer  processed. 


11.  In  selecting  tasks  for  training  some  tasks  may  be  deleted  because: 

a.  They  would  cost  too  much  to  train. 

b.  They  don't  need  to  be  trained. 

c.  Nobody  could  do  them  even  if  they  were  trained. 

d.  They  are  not  tasks  at  all,  but  ..re  elements  of  tasks. 


TW-PPI-3 


Management  inputs  and  approvals  are  an  essential  part  of  tasks  for 

training  because: 

a.  This  "makes  it  official." 

b.  You  are  more  likely  to  get  all  the  resources  you  need  if  you 
get  management  involved. 

c.  The  final  selection  of  tasks  for  training  will  depend  on 
management  resource  and  need  assessments. 

d.  Management  will  rank  order  tasks  for  you  so  you  can  select  them 
for  training. 


MODULE  2 


DIRECTIONS:  U"  lug  the  a\.swer  sheet  or^vided  fur  reccr^'j^ig  your  responses^ 
choose  the  most  appropriate  answe**  for  each  Ite^n.  Begin  with  mj!r4>er  13 
on  your  answer  shy^t.  "" 


13.  A J’M  is  said  to  have  good  "prewictive  validity*'  if: 

a.  it  accurately  predicts  whr.t  tasks  can  be  trained. 

b.  Those  passing  the  JPM  can  perform  the  task  adequately. 

c.  It  predicts  the  validity  of  each  task  in  a job. 

d.  Those  performing  the  task  can  predict  the  measurement  methods 
used, 


14.  If  predictive  validity  Connot  be  assured,  then: 

a.  A JFM  should  have  high  physical  fidelity. 

b.  A JP-M  should  ha'/e  low  physical  fidelity. 

c.  The  task  should  be  measured  by  a JPT. 

d.  Someone  wrote  the  JPM  incorrectly. 


15.  Which  task  would  most  likely  have  a JPM  with  hiqh  predictive  valicity? 

a.  Typing  requisitions 

b.  Dispcoing  of  a live  hand  grenade 

c.  launching  an  IC6M 

d.  Stopping  arterial  bleeding 


16.  A multiple  cask  is: 

a.  A task  among  a sequence  of  dependent  tasks. 

b.  A task  wi*-.h  a number  of  possible  inputs. 

c.  A task  that  is  done  repeatedly, 

A task  with  more  than  one  element. 


*7,  Task  performance  may  be  meacired: 

a.  For  adequacy. 

0.  For  appropriateness. 

c.  By  more  than  one  JPM. 

d,  «11  of  the  above. 


. N 
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16.  All  JPMs  must  ineasuyg: 

a.  A product. 

b.  A process. 

c.  Both  a and  b. 

d.  Either  a or  b,  or  both. 


\m 


JPM  cues  should  be: 

a.  Critical. 

b.  Always  identical  to  on  the  .lob  cues. 

c.  Realistic,  especially  critical  cues. 

d.  Given  without  warnlna. 


20.  JPM  standards  are: 

a.  Applied  to  all  persons  taking  the  test. 

b.  Never  specified  in  advance. 

c.  Necessarily  high. 

d.  Always  objective. 


21.  Which  is  not  a JPM  scoring  procedure? 


a.  Product  rating 

b.  Checklist 

c.  Scales  o-^  performance 

d.  Rando.n  assessment  technique 


JP  " rating  errors  include; 

a.  Er'^rs  of  standard. 

b.  Errors  of  management. 

c.  Errors  of  halo. 

d.  Both  a.  and  c. 


23.  A validated  JPM: 

a.  Has  proven  predictive  validity  and/or  physical  fi  lity. 

b.  Has  demonstrated  Its  cost-effectiveness. 

c.  Is  used  to  find  rating  errors. 

d.  Is  exactly  the  same  as  a verified  JPM. 
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24.  uf'Hs  should  test  performance  under  consitions  that: 

a.  Are  as  bad  as  possible. 

b.  Would  make  a difference  in  performance. 

c.  Are  similar  to  those  in  the  classroom. 

d.  Are  not  to  be  enr.untered  on  the  job. 


£f . ?.  new  ilP.M  should  be  Ud  out: 

p 

d.  On  &>ery  student  to  validate  it.  ' 

b.  I's;  ; tcv’  iob  riold''-'c  to  uf>cover  any  diific'.,-ities.  ! 

c.  Befort  >j.  • Instructu  O >s  Qi''efi. 

d.  To  generate  ro'trac'f' t>ral  o.cjectiv>oi. 


26.  A JPM  checklist  is: 

a.  A list  of  JPMs  yet  to  be  taken. 

b.  An  aid  for  a person  to  use  during  a task. 

c.  A list  of  observable  elements  used  in  rating  performance. 

d.  A device  for  determining  that  JPi;s  are  valid. 


I 
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f 
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MODULE  3 


DIRECTIONS:  Using  the  answer  sheet  provided  for  recording  your  responses, 
choose  the  most  appropriate  answer  for  each  item.  Begin  with  number  27 
on  your  answer  sheet. 


27.  Existing  courses  should  be  analyzed: 

a.  To  avoid  duplication  of  effort. 

b.  To  possibly  save  resources. 

c.  To  use  as  a temporary  expediency. 

d.  All  of  the  above. 


28.  When  analyzing  existing  courses  your  primary  concern  is: 

a.  How  much  money  can  be  saved  by  adoption. 

b.  Whether  it  will  teach  the  required  performance. 

c.  Whether  it  was  developed  in  your  command. 

d.  What  JPAs  are  available. 


29.  Job  analysis  data  for  an  existing  course  may  not  be  acceptable  because: 

a.  It  may  be  obsolete. 

b.  It  may  not  have  been  coPected  correctly. 

c.  Both  of  the  above. 

d.  None  of  the  above. 


30.  In  adapting  an  existing  course: 

a . Adopt  i ts  JPMs  as  wel 1 . 

b.  Examine  its  JP.Ms  for  useability. 

c.  Ignore  the  JPKs  and  write  your  own  to  assure  quality. 

d.  Revalidate  the  course  only  if  the  student  populations  are  different. 


31.  What  must  you  find  out  about  an  existing  course  that  will  indicate 
whether  or  not  the  course  would  be  suitable  for  your  use? 

a.  That  the  course  was  ISD'd. 

b.  All  Phase  I steps  were  done  with  '■  population  similar*  to  your 
intended  population. 

c.  That  the  content  is  the  same. 

d.  That  the  course  has  more  tasks  for  training  than  you  need. 
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MODULE  4 


DIRECTIONS:  Using  thu  answer  sheet  provided  for  recording  your  responses, 
choose  the  most  appropriate  answer  for  each  item.  Begin  with  number  32 
on  your  answer  sheet. 


32,  Which  of  the  following  is  not  a description  of  an  instructional  setting? 

a.  Formal  On-the-Job  Training 

b.  Installation  Support  School 

c.  Memphis  Naval  Air  Station 

d.  Resident  School 


33.  Which  of  the  following  is  a JPA? 

a.  Tool  kit 

b.  Troubleshooting  and  repair  manual 

c.  Repair  record  form 

d.  Exhaust  gas  analyzer 


34.  STEPS  may  eliminate: 

a.  The  need  for  an  instructor's  presence. 

b.  Feedback  to  the  student, 

c.  Self-study. 

d.  All  of  the  above. 


35,  FOJT  would  least  likely  be  used  in: 

a.  Radar  repair. 

b.  Parachuting. 

c.  Clerical  work. 

d.  Jet  engine  overhaul. 


36.  Clustering  lasks  is  done  with  regard  to: 

a.  Conditions  and  constraints  that  would  influence  setting  selection. 

b.  Skill  and  rank  of  the  trainees. 

c.  Personnel  and  equipment  needs  for  instruction. 

d.  All  of  the  above. 


37.  Which  of  the  following  would  reduce  the  chances  of  using  a JPA  without 
training? 

a.  Task  requires  only  a simple  set  of  procedural  steps. 

b.  Task  requires  great  physical  skill. 

c.  Task  if  always  done  in  an  office  with  pencil  and  paper. 

d.  Task  is  only  done  rarely. 
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38.  STEPS  should  probably  not  be  selected  in  training  tasks: 

a.  That  are  primarily  intended  for  individual  instruction. 

b.  Where  close  supervision  is  required. 

c.  Both  a and  b above. 

d.  None  of  the  above. 


39.  Where  large  groups  of  individuals  must  be  taught  the  same  thing  at 
the  same  time,  which  of  the  following  can  often  provide  effective, 
efficient  training? 


a.  Resident  School 

b.  Formal  On-the-Job  Training 

c.  Job  Performance  Aids 

d.  Work  experience 


40. 


A task  may  be  assigned  to: 

a.  Group  instruction. 

b.  Both  RS  and  FOJT. 

c.  Only  JPAs  as  instruction. 

d.  All  of  the  above. 


41.  The  factors  you  primarily  should  consider  when  you  recommend  where 
training  should  be  conducted  are: 


a. 

b. 

c. 
i 

d. 


Training  time,  difficulty,  task  complexity. 

Decay  rate,  supervision,  resource  constraints,  task  conditions. 
Existing  courses,  availability  of  simulators,  number  of 
instructors. 

Climate,  intelligence  of  students,  investment/ return  rate. 


42.  Which  setting  is  likely  to  have  the  most  instructional  resources? 


a. 

JPA 

b. 

STEP 

c. 

RS 

d. 

FOJT 

Which  of 

training 

a. 

JPA 

b. 

STEP 

c. 

RS 

d. 

FOJT 

the  following  is  likely  to  be  the  least  expensive  setting  for 
tasks  which  are  suitable  for  all  settings? 
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MODULE  5 


DIRECTIONS:  Using  the  answer  sheet  provided  for  your  responses,  choose  the 
tnost  appropriate  answer  for  each  item. 


1.  Learning  objectives  (LOs): 

a.  Are  based  on  JPMs. 

b.  Describe  what  students  will  learn  to  do. 

c.  Both  a and  b. 

d.  None  of  the  above. 


2.  LOs  and  Terminal  learning  objectives  (TLOs)  should  be  written  with 
three  parts: 

a.  Action,  noun,  condition. 

b.  Action,  condition,  standard. 

c.  Subject,  action,  standard. 

d.  Subject,  verb,  requi»*ement. 


3.  The  first  step  in  developing  objectives  is  to: 

a.  Prepare  and  document  TLOs. 

b.  List  the  necessary  learning  steps. 

c.  Validate  the  JPMs. 

d.  Determine  student  entry  behavior. 


4.  Which  is  iTot  a criterion  of  LOs  and  TLOs? 

a.  Must  be  stated  in  learner  {not  teacher)  terms. 

b.  Must  specify  an  overt  act  as  evidence  of  learning. 

c.  Must  state  a standard  against  which  student  behavior  is  measured. 

d.  Must  state  precisely  what  steps  the  teacher  must  take  to  prepare 
the  student. 


5.  The  four  learning  categories  are: 


a.  Physical  skill,  information,  attitude,  mental  skill. 

b.  Verbal  skill,  quantitative  skill,  manual  skill,  mental  skill. 

c.  Infomation,  communication,  execution,  resolution. 

d.  Quantitative,  qualitative,  oral,  written. 
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6.  Which  is  a suitable  TLO  "condition”? 

a.  Within  one  second 

b.  Under  water 

c.  Given  an  M16A1  rifle 

d.  Using  a mark  5 transit 


7.  Which  is  not  a type  «f  standard  for  an  LO? 

a.  Referring  to  SOP 

b.  Implying  no  error 

c.  Specifying  what  is  done 

d.  Specifying  product  quality 


8.  Which  action  reveals  attitudes? 


a.  Recalls 

b.  Classifies 

c.  Chooses 

d.  Adjusts 


9.  Learning  analysis  breaks  TLOs  down  into: 

a.  Steps,  elements,  and  motions. 

b.  LOs  and  learning  steps  (LSs)  necessary  for  TLO  mastery. 

c.  The  instructional  procedure. 

d.  Their  fundamental  characteristics. 


10.  In  mental  skills  learning  analysis,  a learning  hierarchy: 


a.  Identifies  subskills  needed. 

b.  Is  of  little  use  since  it  concerns  information. 

c.  Ranks  learning  by  category. 

d.  Simplifies  decision  making  for  the  st'.dent. 


11.  In  information  learning  analysis  you  are  not  concerned  with: 

a.  Listing  facts  the  student  must  learn. 

b.  Developing  memory  devices  for  information. 

c.  Eliminating  from  instructional  consideration  apy  facts  the 
stijdents  already  know. 

d.  Information  v/hich  can  be  "stated". 
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12. 


A physical  skills  learning  analysis  specifies: 


a.  The  amount  of  strength  and  coordination  needed  to  perform  a 
task. 

b.  The  individual  movements  and  sequencing  of  movements  necessary 
to  form  a total  skill. 

c.  The  information  and  mental  skills  required  to  perform  the 
physical  skills. 

d.  All  of  the  above. 


13.  In  attitude  laarning  analyses  you  should: 

?.  Observe  the  attitudes  needed  for  a task. 

b.  List  the  knowledge  needed  to  form  good  attitudes. 

c.  List  the  things  a person  would  do  or  say  to  indicate  the 
desired  attitude. 

d.  All  of  the  above. 


14.  The  purpose  of  the  "conditicns"  part  of  the  objective  is: 

a.  To  allow  analysis  of  a job  product. 

b.  To  describe  the  time  limit  on  task  performance. 

c.  To  define  the  task  performance  process. 

d.  To  describe  the  tools  and  aids  and  the  physical  environment 
present  during  task  performance. 


15.  The  purpose  of  the  "standards"  part  of  the  objective  is: 

a.  To  make  similar  tasks  identifiable. 

b.  To  describe  acceptable  task  processes  and  products. 

c.  To  define  standard  operational  procedures. 

d.  To  describe  the  beginning  and  ending  points  of  a tack. 


16.  How  detailed  a test  should  be  depends  on: 

a.  How  long  the  course  lasts. 

b.  The  use  for  which  it  is  intended. 

c.  The  scoring  method. 

d.  Which  tasks  are  most  important. 


17.  In  using  written  tests  for  non-writing  tasks,  the  main  disadvantage  is: 

a.  The  artificial  initiation  and  termination  cues. 

b.  Cheating  is  possible. 

c.  It  takes  much  longer  to  score. 

d.  The  test  can  never  be  very  accurate. 
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18.  Scores  on  mental  skills  tests  shculd  be  distributed: 

a*  Evenly  across  all  response  <^tions. 

b.  With  distinct  clusterings  of  students’  scores. 

c.  With  most  people  doing  worse  than  average. 

d.  Any  of  the  above  should  be  expected. 


19.  The  problem  with  testing  infomation  is: 

3.  Setting  standards  due  to  lack  of  job  criterion. 
D.  San^ling  due  to  large  amount  of  information. 

c.  Both  a and  b. 

d.  Neither  a nor  b. 


20.  In  testing  physical  skills  you  should  set  TW  test  standards: 

a.  As  high  as  possible. 

b.  Very  low  for  new  learners. 

c.  At  an  Intennedlate  level. 

d.  To  match  the  JPM. 


21.  it  is  best  to  test  attitudes  by; 

a.  Observing  student  behavior  for  evidence 

b.  Using  a polygraph  during  the  TLO  test. 

c.  G1v1r«g  the  student  a nwHIple  choice  test. 

d.  Having  the  student  list  exan^les  of  good  and  bad  attitudes. 

22.  In  which  case  should  standards  be  low  enough  to  be  attainable  and  high 
enough  to  be  an  interesting  challenge? 

a.  TLO  test 

b.  Within  training 

c.  Job 

d.  JPM 


23.  A false  positive  is  exemplified  by: 

a.  A failing  student  who  may  be  able  to  perform  the  task. 

b.  A "yes”  answer  to  a false  question. 

c.  A graduate  marksman  who  can't  adjust  a rifle  sight. 

d.  Affirmation  that  a "false"  response  is  correct. 
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24.  In  order  to  rank  order  students  by  perrcrmance  you  will  need: 

K rt  a distrlbutioi  ’^•'oducing  test 

c I ^core  0®  record. 

d‘  An  ^ ® ranking  test, 

d.  An  extreieJ;  i5'*ge  rutriber  of  students. 

• ■>  In  settin-,  off  scores  remtimber: 

'u  higher  the  better. 

-c  s the  TLO  that  is  most  important, 
c.  A score  of  70^  is  passing. 

V.  Tne  LOs  completely  determine  job  performance. 

26.  The  term  "fidelity"  as  .sed  in  testing  means; 

a.  Test  results  predict  job  results. 

b.  The  degree  to  which  the  ter  situation  "looks  V‘:e"  the  iob 

situation.  uie  juu 

c.  The  honesty  of  the  answers  giveii  concerning  attitudes. 
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MODULE  6 


DIRECTIONS:  Using  the  answer  sheet  proyided  for  your  responses, 
Choose  the  most  appropriate  answer  fcr  each  item.  Begin  with  number 
27  on  your  answer  sheet. 


27.  Which  of  thfe  following  includes  the  sVill?,  information,  and 
attitudes  that  an  individual  possas-es  at  che  time  he  comes  for 
instruction 

a.  Aptitude 

b.  Knowledge 

c.  Entry  behavior 

d.  Baseline  ability 

28.  The  external  systems  requirements  which  state  minimum  entry 
behavior  arc; 

a.  Rank,  security  clearance,  criminal  record. 

b.  Age,  height,  virion. 

c.  Aptitude,  degrees,  language  ability. 

d.  Any  of  the  above. 


In  order  to  verify  or  revise  your  list  of  learning  objectives 
and  their  test  you  should: 

a.  Test  a sample  of  the  population  with  the  LO  ter-,  items  to  determine 
the  degree  of  match  between  the  LOs  and  the  en-L.y  behavior. 

b.  Do  a summative  evaluation  after  the  training  period  and 
then  make  any  needed  improvements. 

c.  Both  A and  B above. 

d.  Neither  A nor  B abo-ve. 


30.  A test  of  students  entry  behavior  may  result  in: 

a.  Further  learning  analysis. 

b.  Deletion  of  some  of  -the  learning  objectives. 

c.  Either  A or  B above. 

d.  Neither  A nor  B above. 


Ins'-ruction  should  begin  at  the  point  that  matches  the  entry 
behavior  of: 

o.  All  prosoective  studencs. 

b.  Most  prospective  students. 

c.  Some  prospective  students. 

d.  None  of  the  above. 


i 


32.  An  entry  test  is  needed-. 

a.  When  deficient  students  will  be  given  remedial  instruction 
before  tr/e  course  begins. 

b.  To  determine  if  incoming  students  abilities  have  increased 
ever  time. 

c.  To  determine  if  some  students  need  only  part  of  the  course. 

d.  When  external  requirements  of  the  system  equal  or  exceed 
those  for  course  entry. 


33.  An  entry  test  compares  to  a pretest  in  that: 

a.  An  entry  test  is  much  longer. 

b.  An  entry  test  contains  only  entry  level  objectives  but 
a pretest  covers  the  course. 

c.  Both  A and  B above. 

d.  Neither  A nor  B above. 


34.  Pretests  can  be  for: 

a.  Internal  course  evaluation. 

b.  Determining  if  student  entry  behavior  is  changing  t/er  time. 

c.  Indicating  that  some  students  should  jump  ahead  rc  sa;a 
time  and  resources. 

d.  All  of  the  above. 


35.  A post-test  may  be: 

a.  An  alternate  form  of  ? oest. 

b.  More  detailed  than  r _5t. 

c.  Either  A or  B above. 

d.  Neither  A nor  B abovt. 


36.  Which  of  the  following  rules  apply  to  sequencing  of  instructions? 


a.  Supportive  objectives  should  be  placed  close  together. 

b.  Independent  objectives  may  be  placed  in  any  order. 

c.  Dependent  objectives  must  be  presented  after  prerequisite 
instruction. 

d.  a,b,  and  c 

e.  a and  c 


i 
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37.  The  purpose  of  sequencing  learning  objectives  is  to ; 

a.  Place  objective  in  an  optimum  sequence  of  presentation, 
b!  Produce  the  most  learning  in  the  shortest  time. 

c.  Both  of  the  above. 

d.  None  of  the  above. 

38.  Sequencing  LOs  with  dependent  relationships  is  simplified  by; 

a.  A good  learning  hierarchy. 

b.  A good  pretest. 

c.  Interview  data  from  jot  incumbents. 

d.  Consulting  the  course  instructors. 


"Fly  an  F5B"  has  a 

a.  Dependent 

b.  Independent 

c.  Supportive 

d.  None  -'f  the  above 


relationship  to  "Fly  an  F4E", 


Sequencing  LOs  which  are  supportive  causes 
done  correctly. 

3 Enthusiasm 
u.  Skill 

c . Dependency 

d.  Transfer 


One  basic  sequencing  rule  is  to  place  two  learning  objectives 
close  together  in  the  learning  sequence  when: 

a.  One  learning  objective  is  dependent  on  the  other. 

b.  There  is  a supportive  relationship  between  the  two  learning 

objectives.  ^ . 

c.  The  learning  objectives  are  independent  of  encn  other. 

d.  The  learning  objectives  are  "common  factor"  objectives. 


42.  Which  of  the  following  is  not  true  of  sequencing? 

a.  Sequencing  is  important  to  low  aptitude  students. 

b.  Sequencing  is  important  to  redundant  materials. 

c.  Sequencing  effects  are  long  range. 

d.  Sequence  is  Important  with  unfamiliar  materials. 
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Sequencing  is  least  important  to  objectives  with: 

a.  Independent  relatiotiships. 

b.  Dependent  relationships. 

c.  Supportive  relationships. 

d . Common  factors . 


Common  element  LOs  generally  should  be  sequenced; 

a.  Toward  the  end  of  the  course. 

b.  Toward  the  beginning  of  the  course. 

c.  With  wide  separation. 

d.  After  all  of  the  dependent  LOs. 


Objectives  may  be  grouped  by: 

a.  Their  relationship  to  each  other. 

b.  Their  relationship  to  the' original  grouping  of  tasks. 

c.  Both  A and  B above. 

d.  Neither  A nor  B above. 


Grouping  of  LOs: 

a.  Should  be  well  done  because  once  decided  on^  the  sequence 
groups  are  unalterable. 

b.  Will  undoubtably  change  the  sequence. 

c.  Is  to  help  clarify  the  program. 

d.  Is  seldom  necessary  for  complex  tasks. 


You  would  develop  a test  of  entry  behavior: 

a.  Based  on  the  learning  analysis. 

b.  Based  on  assumptions  about  the  intended  students, 

c.  Based  on  trials  on  the  intended  students. 

d.  a,  b,  and  c 

e.  a and  c 
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MODULE  7 


DIRECTIONS;  Usinq  the  answer  sheet  provided  for  recording  your  responses, 
choose  the  most  appropriate  answer  for  each  item. 

1.  Learners  should  be  informed  of  TLOs  because: 


It  helps  them  meet  the  objectives. 

It  makes  them  take  pretests. 

It  satisfies  their  natural  curiosity. 
It  prompts  their  activity. 


2.  Choose  the  false  statement. 


a.  Students  learn  faster  through  active  practice. 

b.  Skills  are  improved  during  learning  through  practice. 

c.  Watching  demonstrations  is  effective  practice. 

d.  Retention  of  learning  is  imp''oved  by  practice. 


Which  is  true  of  prompting? 

a.  Cues  or  prompts  before  performance  eliminate  time  wasting  trial 
and  error. 

b.  Cues  can  be  used  before  or  during  practice. 

c.  Both  of  the  above. 

d.  Neither  of  the  above. 


Feedback  is  letting  the  student  know  all  but  which  of  the  following? 

a.  What  he  is  doing  wrong. 

b.  How  he  can  improve  his  performance. 

c.  What  he  is  doing  right. 

d.  Mow  others  are  performing. 


Natural  feedback: 

3.  Is  present  during  learning  and  on  the  job. 

b.  Is  supplied  by  the  instructor. 

c.  Is  present  only  during  learning. 

d.  Must  be  carefully  planned. 


The  most  approoriate  form  of  artificial  feedback  is: 

a.  Complete. 

b.  Immediate. 

c.  Specific, 

d.  All  of  the  above. 
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7.  The  mental  skill  learning  category  has  several  sub-categories.  Which 
of  the  following  is  not  one? 


a.  Classifying-recognizing  patterns. 

b.  Recalling  bodies  of  knowledge 

c.  Identifying  symbols 

d.  Rule  learning  and  using 


8.  Voice  communicating  is  a sub-category  of  the 
category . 


learning 


a.  Mental  skill 

b.  Information 

c.  Physical  skill 

d.  Attitude 


9.  "Distinguish"  and  "monitor"  are  two  of  the  action  verbs  which  might  be 
used  in  a learning  objective  in  the sub-category. 


a.  Detecting 

b.  Recalling  bodies  of  knowledge 

c.  Rule  learning  and  using 

d.  Steering  and  guiding  continuous  movement 


10.  If  a learning  objective  doesn't  seem  to  fit  very  well  into  any  one 
sub-category: 


a.  It  may  not  be  specific  enough. 

b.  It  may  be  better  broken  into  two  objectives 

c.  Bom!  of  the  above. 

d.  Neither  of  the  above. 


11.  Using  learning  guidelines  or  flowcharts  will  result  in: 


a.  Essentially  the  same  learning  activity  selection. 

b.  Two  different  approaches  to  teaching  which  suit- particular  tasks, 
c Confusion,  since  only  one  or  the  other  should  normally  be  used  for 

any  one  course. 

d.  Error,  since  they  must  be  used  together. 


12.  A learning  activity  is: 


a.  Something  done  during  instruction  v.hat  is  specific  to  a particular 
LO. 


b.  Anything  a student  does  during  instruction  or  testing, 

c.  Active  practice  to  help  the  student  learn. 

d.  An  objective  describing  a physical  activity  task. 
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13.  In  order  for  various  LOs  to  be  met: 

a.  The  same  learning  activities  are  required  for  all  LOs. 

b.  Students  must  be  shown  correct  performance  until  they  learn  it. 

c.  appropriate  learning  activities  are  reccnmendeo  ■Jn  order  \o  meet 
.iifferent  objectives. 

d.  The  learning  steps  must  be  repeated  in  the  proper  sequence  L.ntil 
they  are  learned. 


14.  Which  of  the  following  is  not  a consideration  in  specifying  learning 
activities? 

a.  Cost  of  materials  for  media  alternatives 

b.  TLO  conditions 

c.  TLO  standards 

d.  Ability  to  test  performance  of  the  activities 


means: 


15.  In  flowcharting,  the  symbol 


a.  Inputs 

b.  Start 

c.  Outputs 

d.  Decision 


16.  Learning  guidelines  are  used  in  ISD: 

a.  To  improve  learning  effectiveness. 

b.  To  decrease  learning  time, 

c.  To  permit  appropriate  learning  activities. 

d.  a,  b;  and  c. 


17.  Which  of  the  following  is  n^  a general  learning  objective? 

a.  Provide  for  an  adequate  amount  of  trial  and  error. 

b.  Inform  the  learner  of  the  objectives. 

c.  Provide  active  practice. 

d.  Provide  feedback  to  the  learner. 


ESBKraS 
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DIRECTIONS:  Using  the  answer  sheet  provided  for  recording  your  responses, 
choose  the  most  appropriate  answer  for  each  item.  Begin  with  number  18 
on  your  answer  sheet. 

18.  In  selectinv,’  media  for  learning  activities  you  should: 

a.  Identify  stimulus  criteria,  select  media  which  meet  these  criteria 
and  then  use  the  best  two  or  three. 

b.  Identify  stumulus  criteria,  select  media  which  meet  these  criteria 
and  then  use  those  which  are  cheap,  available,  and  apply  easily  to 
your  particular  situation. 

c.  Either  a or  b. 

d.  Neither  a nor  b. 


19.  An  instructional  delivery  system  is: 

a.  Either  printing,  sound,  or  pictures  (movies). 

b.  The  media  pool  selected  for  a course. 

c.  A specified  mode  of  course  operation  based  on  management,  schedule, 
media  and  setting. 

d.  A simulator  or  teaching  machine  and  its  associated  courseware. 


20.  Block  scheduling  of  instruction  is  appropriate  when: 

a.  The  course  content  is  highly  stable  and  the  course  is  expected  to 
have  a long  life. 

b.  Group  interaction  is  a desired  instructional  outcome. 

c.  Both  a and  b. 

d.  None  of  the  above. 


21.  An  important  consideration  when  developing  a system  Master  Plan  is: 

a.  Once  the  plan  is  made,  changes  should  not  be  permitted. 

b.  That  the  plan  need  not  be  too  detailed  if  the  instructional 
setting  is  FOOT. 

c.  That  the  plan  will  serve  as  a basis  for  finding  discrepancies  in 
course  implementation. 

d.  That  the  plan  should  not  duplicate  items  already  included  in  the 
instructional  management  plan. 


22.  An  instructicial  management  plan  is; 

a.  A description  of  course  objectives  and  the  learning  activities 
for  each. 

b.  A work  assignment  for  each  instructor. 

c.  A flexible  course-oriented  cost-accounting  system. 

d.  A set  of  procedures  used  to  assure  a smooth  flow  of  students 
through  the  system. 
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23,  Required  stimulus  criteria  are: 

a.  The  need  for  exciting  learning  activities  associated  with 
critical  learning  objectives. 

b.  The  qualities  or  capabilities  of  a medium  that  are  required  to 
carry  out  the  intent  of  a learning  activity. 

c.  Criteria  of  proper  student  response  during  a learning  activity. 

d.  Used  by  the  instructor  to  select  proper  teaching  methods. 


The  Learning  Objective  Analysis  Worksheet  contains  spaces  for; 

a.  Learning  category  and  activities,  and  media  selection  criteria, 

b.  Media  pool  and  selection  rationale. 

c.  Both  a and  b. 

d.  None  of  the  above. 


Per  amount  of  instruction,  which  costs  least? 

a.  Sound  film  - 16mm 

b.  Television  - 1 inch  video  tape 

c.  Still  photo  prints  - color 

d.  Silent  film  - 8mm 


A medium  which  satisfies;  your  required  stimulus  criteria  may  be 
impractical  for  many  reasons.  Which  of  the  following  is  not  one 
of  these  reasons? 

a.  Time  needed  for  production 

b.  Cost  is  too  high 

c.  Does  not  present  a stimulus  in  sequence 

d.  Medium  is  still  under  technical  development 


A student  self-paced  management  olan: 

a.  Precludes  the  possibility  of  group  meetings. 

b.  Is  preferred  (where  possible)  in  ISD. 

c.  Is  usually  less  effective  than  block  scheduling. 

d.  Is  not  recommended  in  ISD. 


The  IPISD  Manuals  recommend  that: 

a.  Block  scheduling  be  used  when  individual  experiences  are 
expectad  to  vary. 

b;  Group-paced  instruction  be  used  when  appropriate. 

c.  Individualized  instruction  be  used. 

d.  Self-paced  instruction  be  used  when  the  content  changes  frequently. 
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MODULE  9 

DIRECTIONS:  Using  the  answer  sheet  provided  ^or  recording  your  responses, 

choose  the  most  appropriate  answer  for  each  item.  Begin  with  number  30 
on  your  answer  sheet. 

30.  Which  is  often  an  advantage  of  using  off-the-shelf  (existing) 
instructional  materials? 

a.  They  often  cost  less  than  new  materials. 

b.  They  are  often  useable  or  adaptable  to  your  course  quite  quickly. 

c.  They  may  teach  the  task  better  than  the  materials  that  could 
be  made. 

d.  Their  use  may  avoid  unnecessary  duplication  of  effort. 


31.  When  reviewing  existing  instructional  materials  to  decide  whether 
to  use  them  or  produce  new  materials,  a primary  question  is: 

a.  Which  tasks  should  be  taught? 

b.  Are  the  tasks  being  taught  properly? 

c.  How  was  the  course  developed? 

d.  Who  developed  the  course? 


32.  A practical  way  to  select  existing  materials  for  use  is: 

a.  To  eliminate  all  materials  except  those  that  are  appropriate 

to  your  LOs,  learner  characteristics  guidelines,  and  management 
plan. 

b.  Try  out  all  available  materials  and  see  which  of  them  mrk 
best  with  students. 

c.  Both  a and  b. 

d.  None  of  the  above. 


33.  When  collecting  existing  materials  for  review,  you  should  stop  collecting 
when: 

a.  You  have  all  of  the  •naterials  you  can  find  expediently. 

b.  You  find  someUiing  that  will  work. 

c.  You  have  25  or  30  materials. 

d.  None  of  the  above. 


34.  In  considering  whether  to  alter  existing  materials  to  suit  your 
course  needs  better: 

6.  Make  sure  there  is  a long  term  use  for  them. 

b.  Consider  their  condition. 

c.  Remember,  deletion  is  simpler  than  addition. 

d.  All  of  the  above. 


P'l..  s 

fL 

F 
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35.  If  you  find  materials  that  are  satisfactory  for  rnost  students,  but 

1 

difficult  for  some,  you  should: 

a. 

Use  them  only  with  good  students. 

i 

b. 

Prepare  some  remedial  materials  if  possible. 

f 

c. 

Do  not  use  them. 

1 

d. 

Revise  them  to  make  it  simpler 

L. 

36.  The  Learning  Guidelines  are  useful  in  selection  of  existing  materials 

t 

in 

that: 

1 

a. 

Materials  that  violate  the  guidelines  can  be  eliminated 

• 

f 

immediately. 

i 

b. 

The  Guidelines  state  what  the  materials  are  in  advance. 

i 

c. 

They  describe  the  amount  of  learning  guidance  required. 

d. 

Existing  materials  are  consistent  with  learning  guidelines. 

1 

37.  If 

existing  matu.-ials  are  found  that  satis'y  your  course  needs 

fc 

but  don't  confonn  to  your  selected  delivery  system: 

; 

s 

a. 

You  should  use  them  an;A'ay. 

p' 

b. 

Eliminate  them  from  consideration. 

• 

f 

c. 

You  may  be  able  to  modify  them  ino.-^pensively. 

d. 

It  is  unlikely  that  they  will  be  useful  in  any  way  unless 

1 

they  are  "still  visual". 

1 

38.  lisually,  the  existing  materials  collected  bear  what  relationship 

1 

to 

the  course  management  plan? 

1 

1 

a . 

They  will  fit  "as  is". 

f 

i - 

b. 

They  will  require  substantial  modification. 

f 

C • 

They  will  be  unsuitable  altogether. 

1 

1 

d. 

They  will  bear  no  relationship. 

5. 

39.  Revisions  in  existing  materials  consisting  of  ueletions  and/or 

1 

additions  should  be  done  by: 

a. 

The  instructor  concerned. 

i 

b. 

A subject  matter  expert. 

R 

c. 

A media  specialist. 

%■ 

d. 

Any  of  the  above. 

1 

40.  The  principal  aim  of  this  block  in  the  ISO  procedures  is  to: 

- 

1 

a. 

Save  time. 

1 

h. 

Save  money. 

i 

1 

c. 

Avoid  duplication  of  effort. 

1 

d. 

All  of  the  above. 

1 

- - 

- - - - - - - - 
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41.  The  critical  factor  of  success  in  accomplishing  the  work  required 
in  this  block  is: 

a.  The  do’.  I dr  vulue  of  the  selected  materials. 

b.  The  time  required  to  use  the  selected  materials. 

c.  The  student  acceptance  of  the  selecved  materials. 

d.  The  appropriateness  of  the  selected  materials  to  the  course. 


42.  The  final  step  n selecting  materials  is: 


a.  Obtaining  cownand  approval. 

b.  Acquiring  enough  copies  to  meet  local  needs. 

c.  Approval  of  a subject  macter  expert. 

d.  Obtaining  satisfying  results  in  student  tryo-. ts. 


43.  Instruction  need  not  be  prepared  if: 

a.  A platform  lecture  is  to  be  used. 

b.  Formal  On-the-Job  Training  is  to  be  used. 

c.  Both  a and  b. 

d.  Neither  a nor  b. 


44.  "Developing"  materials  should  be  first  tried  on  students: 

a.  As  soon  as  a draft  lesson  is  completed. 

b.  After  it  has  been  approved  by  a subject  matter  expert. 

c.  As  soon  as  there  is  a draft  of  a meaningful  unit. 

d.  After  the  course  is  assembled  as  a whole. 


45.  Such  items  as  the  target  population  description,  testing  information, 
and  instruction  description  should  be: 

3.  Included  in  the  instructor's  guide. 

b.  Made  a part  of  the  instruction. 

c.  Included  in  the  student's  guide. 

d.  Prepared  after  the  first  course  session. 


46.  Which  of  the  following  are  .not  likely  to  be  involved  in  the  develoixsent 
of  instructional  materials? 

a.  Job  incumbents 

b.  Subject  matter  experts 

c.  Media  specialists 

d.  Writers 


47.  Tho  most  valuable  inputs  for  beginning  the  development  of  instructional 
materials  will  be  found: 


a.  In  the  mind  of  the  developer. 

b.  On  the  Learning  Objective  Analysis  Worksheet. 

c.  In  the  front  end  analysis  documentation. 

d.  Ir  the  ^'eedback  fu-m  successful  materials. 


48.  In  preparing  video  materials,  a .<^toryboard  is: 

a.  The  title  or  su' title  print. 

b.  A live  or  recorded  audio  adjunct. 

c.  A pictuia  containing  a written  paragraph. 

d.  A sketch  of  what  is  to  be  shown. 


49.  In  prograrra.ed  instruction,  a frame  is: 

a.  A unit  of  information  and  questions  on  it. 

b.  A single  photograph  or  drawing. 

c.  One  asp'-ct  or  point  of  view  of  a topic. 

d.  A singl.  rule  or  example  of  something. 


50,  An  adjunct  program: 

a.  Is  a different  way  of  teaching  than  any  other. 

b.  Is  a test  and  feedback  technique. 

c.  Uses  special  directions  to  guide  a stude..t  to  specific  parts 
of  existing  instructional  materials. 

d.  Reduces  the  amount  of  teacher  preparation  needed  to  validate 
an  instructional  program. 


51.  In  describing  a JPA; 

a.  Each  step  in  the  procedure  should  co~  'st  of  several  m.ovements 
to  promote  memorization. 

b.  Use  the  shortcuts  tnat  the  experts  use. 

c.  Each  step  should  contain  one  operation  or  movement. 

d.  Each  step  should  have  a description  or  why  a certain  operation 
or  '•ovement  is  performed. 


52.  To  do  a good  job  of  designing  an  FOOT  program  you- will  need: 

a.  A thorough  task  analysis  and  the  help  of  a subject  matter  expert. 

b.  The  resident  school  documentation  for  the  task. 

c.  A media  expert  and  the  results  of  tht  tasK  analysis. 

d.  Time  to  verify  the  design  through  twenty  or  thirty  iterations. 
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53.  Job  Performance  Aids  should  no^  be  used  when: 

a.  The  behavior  sequence  is  long  and  complex. 

b.  Task  delay  tolerance  is  very  ,ow. 

c.  The  tasks  are  rarely  performed. 

d.  The  task  must  be  performed  from  memory. 


54.  Which  of  the  following  is  not  true  of  FOOT  progrr  ? 

a.  STEPS  may  b‘i  effectively  used. 

b.  RS  instruction  should  normally  follow. 

c.  Media  is  rarely  used. 

d.  JPMs  should  be  used. 


55.  The  first  draft  of  instruction  or  materials  should: 

a.  Be  as  complete  as  possible. 

b.  Contain  the  minimum  possible  instruction. 

c.  Contain  as  much  instruction  as  possible. 

d.  Resemble  existing  materials  and  lessons. 


56  The  design  objective  of  instruction  is; 


a.  Design  the  best  possible  instruction. 

b.  Design  average  instruction. 

c.  Design  effective  instruction  at  minimum  cost. 

d.  Design  instruction  most  preferred  by  students. 
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MODULE  10 


DIRECTIONS:  Using  the  answer  sheet  provided  for  recording  your  responses, 
choose  the  most  appropriate  answer  for  each  item.  Begin  with  number  57 
on  your  answer  sheet. 

57.  Validation  of  instruction  should  begin: 

a.  As  soon  as  a meaningful  segment  of  material  is  complete. 

b.  As  soon  as  you  are  reasonably  sure  the  instruction  is  correct. 

c.  When  you  think  you  have  enough  instruction  to  insure  that  the 
desired  behaviors  will  be  learned.. 

d.  As  soon  as  you  get  a clear  idea  of  what  the  instruction  will  be 
like. 


58.  Instruction  that  is  being  validated  should  be  designed  to  be: 


a.  Clearly  acceptable. 

b.  Acceptable. 

c.  Clearly  unacceptable. 

d.  Barely  unacceptable. 


59.  The  most  effective  validation  method  is  to: 

a.  Have  an  expert  observe  the  instruction  as  it  takes  place  and 
tell  you  what  needs  to  be  changed. 

b.  Try  it  out  on  members  of  the  target  population  and  revise  it  until 
it  works. 

c.  Both  a and  b. 

cl.  None  of  the  above. 


60.  During  validation  you  should  collect: 

a.  Only  the  data  you  intend  to  use. 

b.  Roth  student  attitudes  and  test  data. 

c.  All  the  data  you  can  get  from  a trial 

d.  Both  b and  c. 


61.  The  subject  selected  for  an  individual  validation  trial  should  be; 

a.  Reasonably  expert  in  the  subject  matter  area. 

b.  An  individual  Who  has  the  proper  attitude  toward  ISD. 

c.  A member  of  the  target  population. 

d.  Anyone  who  doesn't  know  the  material  yet. 
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By  inspecting  the  operation  of  a course  you  can  noi-_  tell 

a.  What  the  objectives  are. 

b.  That  the  tests  match  the  learning  objectives. 

c.  That  the  instruction  is  valid. 

d.  That  it  is  beinn  taught  well. 


63.  The  effectiveness  of  instruction  as  it  is  being  developed  is  best 
judged  on  the  basis  of: 

a.  Consensus  of  the  developers. 

b.  A subject  matter  expert'?  judment. 

c.  Comparative  analysis  with  successful  courses. 

d.  Trials  on  students. 


#1 


Instruction  is  subjected  to  the  validation  process  because: 

a.  Student  performance  is  only  indirectly  observable. 

b.  It  is  the  only  process  to  insure  that  instruction  works. 

c.  Validation  is  a valuable  documentation  for  the  design  of 
successive  courses. 

d.  It  determines  when  initial  development  of  the  course  is  over. 


65.  A learning  objective  rating  sheet  is  used  to: 

a.  Dete>-mine  the  difficulty  of  learning  objectives. 

b.  Identify  a chart  for  determining  minimum  and  maximum  sample 
sizes. 

c.  Rate  attitudes. 

d.  All  of  the  above. 


Platform  lectures  already  in  use: 

a.  Are  not  subject  to  validation. 

b.  Should  be  revised,  if  needed,  and  validated. 

c.  Are  probably  acceptable. 

d.  Should  be  replaced  by  self-pacing. 


Formative  evaluation  is  a: 

a.  Process  for  rating  instructors. 

b.  Process  for  grading  student  performance. 

c.  Process  for  evaluating  instruction. 

d.  Method  of  test  design. 
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68.  Which  of  the  following  is  the  only  known  way  to  insure  the  effsctivene^iS 
of  instruction? 

a.  Rank  ordering  students 

b.  Validation 

c.  Evaluating  instructors 

d.  None  of  the  above 


69.  Instruction  which  greatly  exceeds  the  requirements  for  effectiveness 
is: 

a.  Readily  detected  by  inspection, 
b Always  shows  up  during  tryout. 

c.  Extremely  difficult  to  detect  by  measuring. 

d.  The  most  important  requirement  for  students. 

70.  In  order  to  insure  that  materials  are  effective  in  a course  setting, 
they  should  be: 

a.  Tried  on  small  groups  first. 

b.  Tried  on  as  many  students  as  possible. 

c.  Tried  out  in  their  final  form. 

d.  Revised  after  each  student  uses  them. 


71.  If  you  establish  a criterion  on  a test  or  performance  measure 
of  90,  this  means: 

a.  That  90%  of  the  students  must  pass  the  course. 

b.  That  90%  of  the  students  must  pass  the  first  time. 

c.  That  90%  is  considered  to  be  a "pass"  for  that  test. 

d.  That  lOX  attrition  is  acceptable. 


72.  If  there  are  five  objectives  in  a course,  an  80%  score  on  the  posttest 
is  acceptable  if: 

a.  All  of  the  errors  are  Isolated  to  a single  objective. 

b.  Errors  are  random. 

c.  80%  of  the  people  pass  it. 

d.  The  designers  have  established  the  80%  criterion. 


73.  V/hether  one  accepts  or  rejects  the  instruction  being  tested  depends  on: 

a.  The  level  of  risk  being  accepted  and  student  performance. 

b.  The  number  of  students  being  tested. 

c.  The  number  of  errors  on  each  test. 

d.  The  criterion  score  for  passing  each  test. 
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74.  Validation  of  lectures  is  difficult  because  lectures: 


3.  Do  not  permit  self-pacing. 

b.  Often  depend  on  outside  exercises. 

c.  Are  for  motivation. 

d.  Affect  student  attitudes. 
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MODULE  11 


DIRECTIONS:  Using  the  answer  sheet  provided  for  recording  your  responses, 
choose  the  most  appropriate  answer  for  each  item. 


The  beginning  of  implementation  of  the  instructional  management  plan 
takes  place: 

a.  After  preparing  part  of  an  internal  evaluation  plan  but  before 
beginning  to  conduct  instruction. 

b.  After  beginning  to  conduct  instruction  and  preparing  an  internal 
evaluation  plan. 

c.  Both  a and  b. 

d.  Neither  a nor  b. 


• m 


The  instructor's  manual  may  have  to  be  supplemented  prior  to  implementing 
instruction  because: 

a.  There  probably  will  not  be  time  to  do  it  later. 

b.  The  learning  objectives  may  have  to  be  changed  to  meet  local  needs. 

c.  The  instructor's  manual  will  not  include  the  internal  evaluation 
plan. 

d.  The  target  population  may  have  changed. 


3.  The  role  of  the  instructor  is  that  of: 

a.  Presenting  the  instruction. 

b.  Managing  the  instruction. 

c.  Both  a and  b. 

d.  Neither  a nor  b. 


4.  The  Instructor  should  pay  particular  attention  to: 

a.  Why  the  learning  objectives  were  selected. 

b.  Why  a particular  delivery  system  was  selected. 

c.  How  the  performance  tests  should  be  administered. 

d.  All  of  the  above. 


5.  The  instructor  should  document  changes  from  the  original  course 
plan  because: 

a.  The  results  of  training  might  not  be  credited  to  the  instruction 
that  actually  occurred. 

b.  He  deserved  credit  for  improving  the  course. 

c.  He  may  have  to  explain  and  defend  his  actions. 

d.  Those  who  conduct  external  evaluation  will  heed  to  know  which 
graduates  can  be  expected  to  perform  well  on  the  job. 


TW-PPV-1 


MODULE  12 


DIRECTIONS:  Using  the  answer  shet t provided  for  recording  your  responses, 
choose  the  most  appropriate  answer  for  each  item. 


1.  The  purpose  of  process  evaluation  is  to: 

a.  Inspect  documentation  of  the  course. 

b.  Compare  planned  accomplishments  with  actual  accomplishments. 

c.  Determine  whether  each  process  is  appropriate. 

d.  Inspect  and  document  the  quality  of  the  outputs  of  each  block. 


2.  The  responsibility  for  finding  out  whether  a course  meets  planned 
objectives  is  assigned  to: 

a.  External  evaluators. 

b.  Internal  evaluators. 

c.  Course  managers. 

d.  Senior  instructors. 


3.  Which  of  the  following  is  least  useful  when  obtained  from  students? 

a.  Performance  data 

b.  Time  data 

c.  Perceived  quality  and  preference  data. 

d.  Instructional  improvement  data 


4.  Which  of  the  following  are  students  least  able  to  rate? 

a.  Instructor's  presentations 

b.  Instructor's  knowledge  of  the  subject 

c.  Instructor's  demonstrations 

d.  Instructor's  i»ppea ranee 


5.  A description  of  an  unusual  occurrance  or  problem  in  the  instruction 
is  called: 

a.  A critical  incident  report. 

b.  A student  rating  report. 

c.  An  instructor  rating  report. 

d.  A report  of  progress. 


/ 
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Page  Ref. 

Question 

Answer 

Page  Ref. 

MODULE  1 

1 

d 

2 

21 

d 

195-199 

2 

b 

3 

22 

d 

202-204 

3 

c 

8 

23 

a 

205 

4 

b 

15 

24 

b 

178 

5 

d 

25 

b 

207 

6 

c 

41 

26 

c 

196 

7 

a 

71 

8 

b 

114 

MODULE  3 

9 

c 

23 

27 

d 

215 

10 

b 

139 

28 

b 

215 

11 

b 

141 

29 

c 

221 

12 

c 

143-8 

30 

b 

225 

31 

b 

MODULE  i 

) 

MODULE  4 

13 

b 

158 

32 

c 

231 

14 

a 

161 

33 

b 

233-4 

15 

a 

159-160 

34 

a 

235 

16 

b 

164 

35 

b 

237 

17 

d 

165 

36 

d 

243-5 

18 

d 

170-171 

37 

b 

248 

19 

c 

179 

38 

b 

20 

a 

180 

39 

a 

259 
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41  b 244-259 

42  c 


43  a 


44  c 
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Question 

Answer 

Page  Ref. 

Question 

Answer 

Page  Ref 

MODULE  5 

1 

c 

1-3 

24 

a 

53-54 

2 

b 

5 

25 

b 

55 

3 

a 

26 

b 

4 

d 

8-9 

5 

a 

12 

MODULE  6 

6 

a 

13 

27 

c 

63 

7 

c 

15 

28 

d 

64 

8 

c 

16-17 

29 

a 

65-66 

9 

b 

18 

30 

c 

67-68 

10 

a 

20 

31 

b 

11 

b 

24-25 

32 

a 

70-72 

12 

b 

26 

33 

b 

72-73 

13 

c 

28 

34 

d 

73-74 

14 

d 

35 

c 

15 

b 

36 

d 

16 

b 

38 

37 

c 

80 

17 

a 

40 

38 

a 

82 

18 

b 

41 

39 

c 

83 

19 

40 

d 

87 

20 

d 

44 

41 

b 

87 

21 

a 

45 

42 

b 

89 

22 

b 

47 

43 

a 

90 

23 

c 

51 

44 

b 

92 
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18.  Instruction  certainly  should  be  revised  if: 

a.  The  established  criteria  are  not  being  met. 

b.  There  is  an  excellent  chance  that  it  can  be  made  more  efficient. 

c.  Both  a and  b. 

d.  Neither  a nor  b. 


19.  The  basis  for  revision  should  be: 

a.  INER  results. 

b.  EXER  results. 

c . Ins tructor  feedback . 

d.  Both  a and  b. 
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a.  Many  instructors  are  not  accustomed  to  instruction  emphasizing 
student  performance  rather  than  instructor  performance. 

b.  They  may  be  required  to  operate  unfamiliar  equipment. during  the  instruction. 

c.  They  must  be  completely  competent  with  tests  and  testing  procedures. 

d.  All  of  the  above. 


7.  One  method  used  to  avoid  the  practice  of  instructors  teaching  the 
tests  is: 

a.  Keeping  tests  secret. 

b.  Having  teaching  and  testing  accomplished  by  different  people. 

c.  Having  tests  scored  by  different  people. 

d.  None  of  the  above. 


8.  During  internal  evaluation,  the  instructor  needs  to  document: 

a.  Instructor  problems,  student  questions,  graduates'  performance. 

b.  Changes,  times,  graduates'  performance. 

c.  Times,  student  problems,  instructor  problems. 

d.  The  instructor  is  not  required  to  document. 


9.  The  kinds  of  data  collected  for  evaluation  are: 


a.  All  hard  and  soft  data. 

b.  Hard  data  only. 

c.  Soft  data  principally. 

d.  All  data  that  are  relevant  and  can  be  r.,adily  used. 


10.  When  the  EXER  indicates  a clear  deficiency  in  student  performance  in 
the  field: 


a.  The  instruction  must  be  revised. 

b.  The  tests  must  be  revised. 

c.  Both  tests  and  instruction  must  be  revised. 

d.  Revision  will  depend  on  the  immediate  needs  of  the  field  user. 


11.  System  revision  is  usually  undertaken  by: 

a.  Those  responsible  for  the  initial  development  of  Instruction. 

b.  The  normal  quality  control  function. 

c.  Those  responsible  for  testing. 

d.  Those  who  are  directly  related  to  areas  requiring,  revision. 


N ' r4.».«iini„i»»)», 
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12.  The  lnterr.c.1  evaluator ■^  responsibility  is  to: 

a.  Determine  which  students  should  pass  and  which  should  remain  Tor 
additional  training. 

b.  Determine  if  the  instruction  has  met  the  requirements  as  measured 
by  the  tests. 

c.  Both  a and  b. 

d.  Neither  a nor  L, 


13.  The  primary  purpose  of  external  evaluation  is  to: 

a Determine  if  the  learning  objectives  are  being  met,  and  if  not, 
why . 

b.  Permit  an  impartial  evaluation  of  the  course  content. 

c.  Ensure  that  students  do  not  graduate  until  they  have  received 
s uf f i ci ent  trai ni ng . 

d.  Determine  if  graduates  can  d*.  the  job  for  which  they  were  trained. 


14.  External  ev  iuation  is  donducted: 

a.  Before  the  impiementatjon  of  the  instruction. 

b.  During  imp jementation  of  the  Instruction. 

c.  Both  a and  b. 

d.  Neither  a nor  b. 


15.  Exto-nal  evaluation  data  should  be  collected: 

a.  Before  ^raining  begins. 

b.  Just  before  graduation. 

30  - 90  days  after  graduation, 
d.  One  year  after  graduation. 

16.  Results  of  (External  evaluation  might  be  a recommendation  for; 

a.  Changes  in  che  instruction. 

b.  Chang  : in  the  job  structure;  that  is,  rake  sure  the  acti-al  job 
matcF  the  job  description. 

c.  Both  a and  b. 

d.  Neither  a nor  b. 


17.  The  ISD  process  ends  when: 

a.  Th“  trainees  can  meet  the  learning  objectives 

b.  The  'jTaduates  p^irform  adoqu'*tely  on  the  job 

c.  Both  a and  d. 

d.  Neither  a nor  t. 
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5 

6 
6 
9 
9 

11 

11 

13 

17 
20 

24 
2 

25 

18 
12 

4 


107-123 

125 

138 
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Page  Ri 

22 

d 

105 

23 

b 

107-8 

24 

c 

109 

25 

c 

25  & : 

26 

c 

27 

b 

124-6 

28 

b 

124-5 

29 

c 

139 

MODULE  9 

30 

31 

32 

33 

34 

35 

36 

37 

38 

39 

40 

41 

42 
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6.  If  students  score  higher  on  the  posttest  than  on  the  pretest: 

a.  The  instruction  is  effective. 

b.  The  instruction  is  not  effective. 

c.  The  test  is  reliable. 

d.  None  of  the  above. 


7.  An  individual  takes  a test  on  two  occasions.  He  is  likely  to: 

a.  Score  higher  the  second  time  if  he  had  a very  high  initial  score. 

b.  Score  lower  the  second  time  if  he  had  a very  high  initial  score. 

c.  Score  low  the  second  time  if  he  had  a low  initial  score. 

d.  None  of  the  above. 


8.  External  evaluation  data  is  collected  from: 

a.  The  Occupational  Data  Banks. 

b.  Job  incumbents  and  supervisors. 

c.  DOS  tests . 

d.  Instructors. 


9.  The  purpose  of  the  process  of  revision  in  ISD  is  to: 

a.  Update  job  analysis  data. 

b.  Update  learning  objectives  and  JPMs. 

c.  Control  the  match  between  resident  and  non-resident  instruction. 

d.  Update  the  instruction  so  that  it  meets  the  current  needs  of  the 
field  or  fleet. 


10.  An  example  of  "soft"  Internal  evaluation  data  is: 

a.  The  results  of  a JPM. 

b.  The  results  of  a paper  and  pencil  test  for  the  task. 

c.  A trainee's  opinion  as  to  whether  he  could  perform. 

d.  All  of  the  above. 


11.  Data  from  attitude  or  opinion  surveys  is: 

a.  Not  as  reliable  as  test  data. 

b.  Of  no  value  unless  supported  by  test  data. 

c.  More  reliable  than  test  data  if  secured  from  appropriate  sources. 

d.  Valid  but  not  reliable. 
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MODULE  10 


248  & 275 


262-6 
222-3 
224-5 
234  & 240 


257-8 


282- 4 

283- 4 
288-295 
286  & 289 


296-7 


283-5 
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1 a 2-4 

2 c 5-6 

3 c 30-31 

^ c 34-35 

5 a 35-37 

6 d 7-8 


7 b 33 

8 a 

9 d 

10  d 92 

11  d 115 
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8 
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Page  Ref. 

Question 

Answer 

Page  Ref. 

d 

16 

10 

c 

4 

b 

11 

a 

4 

d 

12 

b 

39-40 

b 

33 

13 

d 

64 

a 

29 

14 

d 

64 

d 

46-47 

15 

c 

68 

b 

48-49 

16 

c 

81-82 

b 

17 

d 

83 

d 

18 

a 

97-98 
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STUDENT  PROFILE  QUESTIONNAIRE 


1. 

Name  (Last,  First,  MI) 

2. 

Rank  (if  applicable) 

3. 

Social  Security  # 

• 

4. .Age 

5. 

Branch 

6. 

DOS 

7. 

Job  Title/Position 

8 . 

Total  Years  in  Service 

9.  Total  Years  in  Current 

Regular  Assignment 

10.  Assignment/ Address/Phone  


I 

- - - - — - — - I 

I 

! 

11.  Number  of  Years  of  Formal  Schooling | 

I 

12.  Circle  highest  degree  completed:  H.S.  AA  BA  MA  EdD 

AS  BS  MS  PhD 

13.  Academic  major  (if  applicable):  BS 

MS 

PhD 

14.  Have  you  ever  had  a course  in  Educational  Technology  (ET)? 

Yes  No 


I 


i 


student  Profile  - page  2 


15.  Will  you  be  directly  engaged  in  some  phase  of  ET  when  you  return  to 
your  regular  assignment? 

Yes  No  Don't  Know 

16.  Do  you  have  any  plans  for  continuing  your  formal  education?  If  so, 
please  indicate  what  those  plans  are. 

Yes  No  Don't  Know 


17.  What  areas  of  ET  most  interest  you?  Rank  order  3 areas. 


1st 

2nd 

3rd 

18.  Are  you  a product  of:  Service  Acadetny 

ROTC 

OCS 

Other  (Specify) 


19.  Do  you  plan  on  making  the  service  your  career? 


Yes 


No 


171 


Job  Pre-file  Form 


The  Job  Profile  Form  is  used  to  assist  the  Enabler  and  the  student 
in  setting  the  students'  criteria  for  each  workshop  module.  This  form 
is  filled  out  by  the  student  in  concert  with  a workshop  Enabler.  An 
Enabler  assists  the  student  in  filling  out  the  form  to  insure  that  clear 
definitions  of  the  functional  statements  are  communicated.  The  student 
is  to  check  the  appropriate  box(es)  for  each  functional  statement. 


A general  rule  of  thumb  is,  if  the  student  performs,  is  responsible 
for,  or  supervises  personnel  who  perform  50%  or  more  of  the  functions 
within  a given  module,  his  criterion  for  that  module  should  be  set  at 
100%,  due  to  the  interrelationship  between  functions  and  assignments 
within  a given  module.  The  actual  criteria  set  within  your  workshop 
will  be  dependent  upon  local  SOP. 


JOB  PROFILE 


You  perform  this  function— 

You  supervise  personnel  who  perform  this  function- 
You  are  ultimately  responsible  for  this  function— 
You  are  disassociated  from  this  function 


FUNCTIONS 


1.  Describe  jobs  by  giving  their  definitions 

2.  Validate  tasks  lists 

3.  Specify  the  conditions,  standards,  cues 
and  elements  of  each  task 

4.  Select  tasks  for  training 

5.  Develop  Job  Performance  Measures  (JPM's) 

6.  Write  Administrative  Instructions  for  JPM's 

7.  Analyze  existing  courses  for  the  purpose  of  task 
analysis 

8.  Select  Instructional  Settings 

9.  Write  Objectives 

10.  Perform  Learning  Task  Analyses 

n.  Write  Test  Items 

12.  Prepare  entry  tests,  pre-posttest,  and  within  course 
tests 

13.  Set  student  entry  levels 

14.  Sequence  and  structure  the  course  objectives 

15.  Classify  I :arning  Objectives  in  terms  of  types 
of  Learning 

16.  Specify  learning  guidelines  and  activities 
which  are  to  take  place  in  the  instructional 
setting 


MODULE 

8 

9 


You  perfonn  this  function 

You  supervise  personnel  who  perform  this  function 
You  are  ultimately  responsible  for  this  function- 
You  are  disassociated  from  this  function 


FUNCTIONS 

17.  Select  instructional  media  and  methods 

18.  Develop  total  plan  for  implementing 
instruction,  e.g.  facilities,  instructor 
personnel,  support,  etc... 


19.  Review  and  select  existing  materials  for  use 
in  courses 

20.  Develop  instructional  materiil 


21.  Develop  instructors  Guide 


10  22.  Validate  the  effectiveness  of  the  course 


23.  Train  instructors 

11 

24.  Present  instruction 


25.  Collect  evaluation  data  on  the  effectiveness 
of  courses 

12  26.  Collect  evaluation  data  on  the  performance  of 

students  in  the  field  following  training 


27.  Revise  the  instructional  system  based  on 
internal  and  axternal  evaluation 
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Technical  Workshop  Progress  Sheet 

The  Technical  Workshop  Progress  Sheet  is  used  to  follow  the  progress 
of  each  student  through  the  Workshop.  This  form  is  filled  out  by  the 
Enabler.  Space  is  made  available  for  recording  data  and  assignment  for 
each  module.  This  fosm  can  be  filled  out  on  a module  or  phase  basis 
depending  on  the  method  used  to  collect  relevant  data.  After  the  results 
the  Phase  pretests  are  compiled  and  the  Enabler  enters  these  scores  on 
the  form  for  each  module.  Using  the  Job  Profile  Form,  the  Enabler  assigns 
the  student's  criterion  for  each  module.  An  assignment  for  each  module  is 
made  based  on  the  student's  criterion  and  his  pretest  score  for  that  module. 

If  the  assignment  requires  reading  and  a posttest,  the  posttest  score  '!s 
entered  in  the  appropriate  space.  The  product  evaluation  column  is  used 
to  reco>"d  the  successful  completion  of  the  module  exercises. 

A sample  of  a partially  completed  form  follows: 

The  student  data  for  this  form  is  shown  here:  j 

i 


Modules 


1 

2 

3 

4 — 

-12 

Pretest  sco.es 

50 

75 

20 

80 

10 

Criterion  (JPF) 

100 

100 

100 

0 

0 
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TECHNICAL  WORKSHOP  PROGRESS  SHEET 


+-> 

OJ  (/) 

•a  o 
o $-  o 
Tta.  tn 


Assignment 


m 40  +J 

<u  <u  u <o 

f—  40  QJ  3 3 

3 4->  S-  *0  •— 

"D  Ifl  O O lO 

O O O S_  > 
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50  100  Read  Blocks  I.l  and  1.2  of  the  ISO 

Manuals  and  take  Module  1 posttest 


75  100 


20  100 


Read  Block  1.3  of  the  ISO  Manuals  and 
take  Module  2 posttest 

Read  Block  1.4  of  the  ISO  Manuals  and 
take  Module  3 posttest 

Discuss  missed  items  with  Enabler  or 
reread  appropriate  section  (judgemental] 

Read  Block  1.5  in  ISD  Executive  Sumnary 
or  equivalent  in  manuals 


Read  Blocks  V.1-V.3  in  ISD  Executive 
Summary  or  equivalent  in  manual s . 


Product 

Evaluation 


DATA  WORKSHEET 


LEARNING  OBJECTIVE  ANALYSIS  WORKSHEET 


Loarniriq  Category 


COMPLEXITY  cmVERIA 

DiMicult  Motor  Acts—— 

Smooth  Motor  Performance  at  end  of 
Training. 

STIMULUS  CRITERIA 
Viwjtl  Form 

Alphinumerie^ 

Picttirial,  Plane 
Lin^i  Conatruction,  Plena 

Object,  Sol  id 

Full  viiuel  environment 

Vlauiil  Movement 

Still j 

L.imited- — 

Full^ — - 

Visual  Spectrum 

Black  and  White 

Gray  Scale 

Color- 

Scale 

Exact  Scala 

Audio 

Voice  Sound  Range— 

Full  Sound  Range — — 

Ambient  Soundi^ 

Other 

Tactile  Cue*- 

Internal  Stimulus  Motion  Cues- 

External  Stimulus  Motion  Cue*——. 

Fin*  movement  manipulative  Acts 

Broad  Movement  manipulative  Acts 

TRAINING  SETTING  CR ITER U 
Individual  Trainee  or  teem  training  at  a 

Fixed  Location — 

individual  Trainees  with  simultanc  ;us 
instruction  or  many  locations  — 
Indivklual  Trainee  or  teem  training  with 
Itxiependent  Instruction  at  any 

Location 

Individual  Trainee  on-th*-j,tb 

Smell  Group 

Large  Group  at  Single  Location  

Team  Setting  

ADMINISTRATIVE  CRITERIA 
Site  of  Courseware  Development 

Local  

Cemral 

Magnitude  of  Acquisition  Cost 

* 

High ■ 
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IPISD Workshop  Director's  Checklist 


1.  Identify  the  departments  and  agencies  within  the  local 

command  that  perform  functions  similar  to  the  ISO  procedures. 

_ 2.  Interview  the  supervisors  of  these  departments  and  agencies 
to  determine  the  needs  of  the  command  with  respect  to  the 
implementation  of  ISD  procedures. 

3.  Iden:ify  workshop  content  options  most  appropriate  for  students 
from  the  various  agencies. 


4.  Identify  potential  workshop  enablers. 

5.  Conducv.  workshop  and  administrative  training  for  potential 
enablers. 

6.  Select  most  qualified  personnel  to  serve  as  erablers. 

7.  Develop  Dverall  schedule  of  activities  for  the  workshop. 

8.  Request  ciassroom  and  equipment  required  for  the  slide-tape 
presentations. 

9.  Request  individual  study  areas  for  students. 

10.  Obtain  the  requirad  number  of  IpiSD'  Manuals,  Student  Workbooks, 
p'-etests/posttests,  and  forms  for  the  des-^gn  and  development  of 
instructional  materials. 

11.  Obtain  all  miscellaneous  materials  required  for  the  worksnop 
(pencils,  paper,  push  pir,,  flip  chart  paper,  staples, 

scotch  tape,  masking  tape,  felt  markers,  chalk,  eraser,  easel). 
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12.  Request  subjects/ trainees  for  the  tryouts  of  student-developed 
instructional  materials. 

13.  Confirm  previously  selected  workshop  content  options 
through  Interviews  with  students  and  evaluations  of  their 
background  and  experience. 

14.  Provide  students  with  a schedule  of  the  tim  and  location  of 
Enabler  availability. 

15.  Conduct  dally  meetings  with  the  Enablers  to  Insure  that  the 
students  are  meeting  their  objectives  according  to  the 
selected  option. 

16.  Develop  data  collection  plan  for  workshop  fo11ow>up  study. 
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